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FOREWORD 


On*  of  th*  goal*  of  th*  Leadership  and  Management  Technical  Area  is  the 
development  of  better  and  more  valid  organizational  effectiveness  diagnostic 
instruments  that  Organizational  Effectiveness  Staff  Officers  (OESOs)  could 
ug*.  This  report  presents  a  compilation  and  comparison  of  extant  work  en¬ 
vironment  questionnaires  in  use  in  the  military  and  private  industry.  In 
addition,  the  review  presents  the  relationships  which  have  been  identified 
between  Army  unit  effectiveness  and  satisfaction  measures  and  the  work  en¬ 
vironment  questionnaire. 

This  report  was  prepared  by  McBer  and  Company  under  Army  Project 
2Q163731A792.  The  Deputy  Chief  of  Staff  for  Personnel  is  the  sponsor. 
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BRIEF 


Requirement : 

To  compile  a  guide  for  use  in  developing  work  environment  assessment 
instruments.  This  guide  should  be  a  "how-to-do-it"  guide  for  OESOs  and 
researchers  interested  in  Army  unit  effectiveness. 


Procedure : 

Three  separate  steps  were  taken  in  the  development  of  this  guide: 

(1)  a  review  of  existing  military  and  civilian  work  environment  and 
organization  climate  questionnaires; 

(2)  interviews  with  representative  Army  combat  arms  and  support  en¬ 
listed  personnel;  and 

(3)  identification  of  empirical  measures  of  Army  unit  effectiveness. 


Findings : 

Thirty-eight  military  and  civilian  work  environment  questionnaires  were 
reviewed  in  depth.  These  instruments  were  reviewed  following  a  taxonomy 
which  includes:  (1)  the  theoretical  construction — to  include  scaling,  item 
content,  item  comprehensibility,  response  set,  and  style;  (4)  instrument 
reliability;  (5)  instrument  validity;  and  (6)  intervention  utility. 


Utilization  of  Findings: 

This  guide  will  serve  as  a  useful  resource  for  any  OESOs  and  researchers 
who  develop  and  tailor  work  environment/organizational  climate  questionnaires 
to  the  Army* 
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Z.  REVIEW  OF  WORK  ENVIRONMENT/ORGANIZATION 
CLIMATE  INSTRUMENTS 


More  than  3,000  "instruments  for  assessing  human  behavior"  are  listed 
in  reviews  of  this  literature  (Lake,  Miles,  &  Earle,  1973;  Pfeiffer  fi  Hes- 
lin,  1973;  Robinson  fi  Shaver,  1976}  Chun,  Cobb,  &  French,  1976).  These 
sources  do  not  include  many  unpublished  instruments  developed  by  human  re¬ 
lations  consultants  and  military  researchers  for  use  in  organisational 
interventions . 


Review  Criteria 

The  first  task  in  reviewing  existing  work  environment1  questionnaires 
was  to  develop  criteria  for  instruments  to  be  examined.  The  following  cri¬ 
teria  were  used. 


Organisation  as  Unit  of  Diae 


Instruments  that  did  not  measure  aspects  of  organisational  functioning 
were  not  reviewed.  Four  types  of  instruments  were  excluded  under  this 
criterion > 


1.  Personality  tests.  Measures  of  individual  traits,  characteristics, 
intelligence,  attitudes,  and  the  like  (e.g.,  the  Minnesota  Multiphasic  Per¬ 
sonality  Inventory,  or  the  California  Personality  Inventory)  were  excluded 
as  being  clearly  not  directly  related  to  organisational  variables. 

2.  Occupational  preference  inventories.  Measures  of  individual  work 
preferences  (e.g.,  the  strong-Campbell  Vocational  Inventory,  Kuder  Prefer¬ 
ence  Teat,  or  Navy  Vocational  Interest  Inventory)  were  excluded  as  being 
primarily  smaaures  of  individual  attitudes,  not  variables  of  organizational 
functioning ,  although  respondents'  perceptions  of  organizational  variables 
were  included. 


3.  "Group  personality"  tests.  A  number  of  instruments  (e.g..  Stock  & 
Thelen's  "Reactions  to  Group  Personality  Test,"  Bales'  "Interaction  Process 
Inventory,”  Schuts's  FZRO-B)  have  a  quasi -organizational  focus  in  that  they 
attempt  to  measure  the  ways  individuals  will  react  in  work  groups.  These 
instruments  do  not,  however,  measure  aspects  of  the  organization  which  im¬ 
pact  on  member  behavior. 


4.  "Managerial  style"  questionnaires .  These  instruments  (e.g., 

Blake  4  Mouton's  GRID,  Reddin’ s  Management  Style  Diagnosis  Test,  Blanchard  & 
Mersey's  LEAD  8elf /Others,  Hall's  Leadership  Appraisal  Survey,  Hemphill  & 
Coon's  "Leader  Behavior  Description  Questionnaire”  (Campbell,  Fiedler's  LPC 
Scale,  Vroom  4  Yetton's  "Authoritarian-Consultative  Continuum”)  measure 


^The  terms  "work  environment"  (WE)  and  "organizational  climate”  (OC) 
will  be  used  interchangeably  in  this  report. 
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supervisors'  propensities ,  behaviors,  or  managerial  practices  in  dealing 
with  their  subordinates.  While  managerial  practices  are  definitely  a  com¬ 
ponent  of  work  group  members'  environment  or  climate,  these  instruments 
measure  this  variable  from  supervisor  self-reports  rather  than  from  sub¬ 
ordinate  perceptions.  Managerial  style  questionnaires  therefore  resemble 
"group  personality"  tests  in  measuring  how  people  think  they  will  behave  in 
organisational  settings,  rather  than  their  behavior  as  an  observable  var¬ 
iable  of  organizational  functioning. 


Survey  Response  Format 

Instruments  that  subjects  could  not  answer  by  responding  to  closed- 
ended  questions  were  not  reviewed.  Excluded  under  this  criterion  were 
structured  interviews,  group  sensing  and  observation  techniques  (e.g., 
the  group  "interaction  process  analysis"  methods  of  Bales,  1950,  1970; 
Stock  &  Thelen,  1958;  and  the  organizational  observation  methods  of  Levin¬ 
son  et  al.,  1972;  Argyris,  1970;  and  Mahler,  1974). 


Organizational  Process  vs.  Satisfaction 

Certain  "pure"  satisfaction  questions  were  not  reviewed  for  two  reasons. 
First,  a  comprehensi ve  review  of  these  instruments  was  recently  completed  by 
ARI  (Motowidlo  et  al.,  December  1976).  Second,  as  will  be  discussed  later, 
both  organizational  intervention  theorists  and  Army  OESO  practitioners  recom¬ 
mend  that  satisfaction  be  considered  an  outcome  of  organizational  function¬ 
ing,  and  that  work  environment  questionnaires  should  focus  on  organization 
structure  and  process  variables  which  account  for  satisfaction  and  produc¬ 
tivity  outcomes,  rather  than  feelings  of  satisfaction. 


Statistical  Analysis 

Certain  questionnaires  which  did  not  report  statistical  data  on  instru¬ 
ment  reliability  or  validity  were  excluded  if  the  interviewers  were  unable 
to  interpret  the  meaning  of  instrument  results  in  the  absence  of  this 
information. 


Military  Pse 

Any  instrument  used  in  the  Tunny,  Navy,  Air  Force,  Marine  Corps,  or 
other  military  organization  was  reviewed,  whether  or  not  is  satisfied  the 
preceding  two  criteria. 

Table  1  summarizes  the  sources  and  construction  characteristics  of 
instruments  reviewed  in  depth.  A  list  of  the  instruments  and  instrument 
sources  examined  is  found  in  Appendixes  A  and  B. 
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Footnotes  are  found  at  end  of  table. 


Number  Number  of  Content  Descriptive 
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Review  Taxonomy 


Figure  1  presents  an  outline  of  the  points  used  to  assess  each  instru¬ 
ment  selected  for  review.  This  section  discusses  these  points  in  detail  in 
order  to  provide  QESOs  with  a  checklist  of  criteria  and  examples  for  use  in 
developing  future  Any  organization  climate  instruments. 


The  instrument  name  used  is  that  recorded  on  the  title  page  of  the 
instrument  itself  or  referred  to  in  the  literature  accompanying  it.  Other¬ 
wise  the  instrument  is  referred  to  by  the  name  of  its  author  (if  given)  or 
the  organization  in  which  it  was  used  if  no  author  is  listed. 


Use 

Zf  the  instrument  was  used  in  a  military  organization  (H) ,  the  service 
branch  was  indicated  by  Army  (A) ,  Navy  (N) ,  Air  Force  (AF) ,  or  Marine  Corps 
(MC).  Instruments  from  other  organizational  environments  were  noted  as  be¬ 
ing  private  business  and  industry  (B) ,  educational  institutions  (E) ,  gov¬ 
ernmental  agencies  (G) ,  health  care  organizations  (H) ,  and  "other"  (0)  en¬ 
vironments  such  as  communities  and  prisons.  Sixteen  military  and  22 
civilian  instruments,  a  total  of  38,  were  reviewed  in  depth. 


References 

References  concerning  the  basis,  interpretation,  reliability  and  valid¬ 
ity,  and  use  in  organizational  interventions  and  replications  studies  were 
identified  and  reviewed  where  available.  Many  instruments  lacked  published 
references,  or  the  cited  references  were  out  of  print  or  untraceable.  Con¬ 
clusions  drawn  about  certain  instruments  should  be  considered  as  being  "to 
the  best  of  the  authors'  knowledge,"  since  undiscovered  source  materials 
may  provide  information  listed  as  "not  reported"  in  this  review. 


Basis 

Instruments  were  classified  as  having  one  of  three  bases: 

1.  Formal  theory  (hypothesis  or  model) .  Many  instruments  were  based 
on  an  identifiable  theory  of  organizational  behavior  traceable  to  a  specific 
author  or  authors.  For  example,  the  most  common  theoretical  basis  for  the 
instruments  reviewed  included  the  "Systems  4"  ideas  of  Likert  (1961)  and 
the  "4-factor  theory"  of  Bowers  and  Seashore  (1966) .  Instruments  based  on 
this  approach  included  the  Army  General  Organizational  Questionnaire,  the 
Navy  Human  Resource  Management  Questionnaire,  the  University  of  Michigan/ 
Institute  of  Social  Research  Survey  of  Organizations.  Numerous  derivative 
instruments  use  items  taken  from  the  Likert/Bowers  and  seashore  research. 
Other  instruments  were  found  to  be  based  on  the  theories  of  Mas low,  Herz- 
berg,  McClelland,  the  Ohio  State  leadership  studies,  and  the  like. 
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Figure  1.  Work  Environment  Questionnaire  Instrument  Review  Taxonomy. 

X.  Instrument  Name 
XX.  Use 

A.  Military  (M) 

1.  Army  (A) 

2.  Navy  (N) 

3.  Marine  Corps  (MC) 

4.  Air  Force  (AF) 

B.  Industry  (B) 

C.  Education  (E) 

D.  Government  (G) 

E.  Health  (H) 

F.  Other  (communities,  prisons,  etc.)  (O) 

XIX.  References 

XV.  Basis 

A.  Academic  theory  (model,  hypothesis) 

B.  "Grounded"  theory  (interview,  group  sensing,  etc.) 

C.  "Shotgun  empiricism"  (source  of  items,  rationale) 

Basic  relevance,  congruence,  face  validity  for  military 
organizations 

V.  Organization  Component  Assessed 

A.  Supervisor*  teacher,  hospital  doctors/staff,  prison  warden/ 
guards  (managerial  practices  or  "style”) 

B.  Job  (task  activity  of  organization) 

C.  Subordinates:  also  students,  patients,  prisoners,  etc. 

1.  Individual  personality 

2 .  Peers 

D.  Organization 

1.  Norms:  standards,  conformity,  etc. 

2.  Policy /procedures 
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Figure  1  (Continued) 


E.  Exogenous  (general  economy,  culture,  competition,  suppliers , 
legal,  regulatory,  political) 


F.  Outcome 

1.  Performance 

2.  Satisfaction 


VI.  Variables  (matrixed  by  organizational  component  assessed) 

VIZ.  Item  Construction 

A.  Scaling 

1.  Anchors 

2 .  Format 

(a)  Likert 

(b)  semantic  differential 

(c)  forced  choice 

(d)  number:  forced  ranking,  distribution  of  points 

(e)  difference 

(f)  other  (operant,  free  response,  etc.) 

3.  Scale  range 

4.  Neutral  points:  midpoint  (odd  number)  vs.  no  midpoint  (even 
number) 

B.  Item  Content 

1.  Cognitive 

2.  Affective  evaluation  (E) 

3 .  Descriptive  (D) 

4 .  Behavior 

C.  Item  Comprehensibility 

1.  Reading  level 

2 .  Item  length 

3.  Semantic 

4.  Syntactics 

5.  Appearance:  response  format 

6.  Undimensionality 

D.  Response  bias:  response  sets  and  response  styles 
1.  Response  set 

(a)  consistent  positive:  "halo"  effects 

(b)  item  order  and  grouping 
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Figure  1  (Continued) 


2 .  Response  styles 

(a)  acquiescence 

(b)  social  desirability:  distancing 

(c)  understatement  (cautious)  responses 

(d)  extreme  (dominant)  responses 

(e)  inconsistency  ("lie")  responses 

E.  Item  and  Instrument  Rejection 

1.  Length 

2.  Interest,  salience,  and  "face  validity" 

3.  Incomprehensibility 

4 .  Threat 

VIII.  Instrument  Reliability 

A.  Sample 

B.  Scale  (split  half) 

C.  Test-re test 

D.  Factor  j 

IX.  Instrument  Validity 

A.  Construct:  "face,"  content,  convergent  validity 

B.  Criterion:  concurrent,  predictive — "causal  flow  model”  statistics 
(cross-sectional  vs.  longitudinal:  time  lag  correlation  designs) 

X.  Intervention  Utility 

A.  Administration 

1.  Length:  number  of  items  and  administration  time 

2.  Administration  instructions 


(a)  rapport:  explanation 

(b)  confidentiality 

(c)  motivation:  "What's  in  it  for  you" 

3.  Resources/supplies  required 

B.  Processing 

1.  Scoring:  hand,  computer  punch,  opscan 

2.  Analysis:  hand,  computer 

3.  Resources  required:  data  processing  equipment,  etc. 

4.  Turnaround  time 
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Figure  1  (Continued) 


Feedback:  Motivation  and  direction  for  change 

1.  Nunber  of  variables  (acre  than  "7  t  2") 

2.  Format 

(a)  simplicity 

(b)  visual  impact 

(c)  motivating  comparison:  actual-ideal,  normative,  achieve¬ 
ment,  absolute 

(d)  specific  directions  for  change  indicated 

3.  Salience 

(a)  clear  link  to  performance  and  satisfaction  criteria 

(b)  face  validity 

(c)  no  normative  bias 

4.  Consultant/client  training/level  of  expertise  required  to  use 
instrument,  make  sense  of  feedback 

Evaluation 

1.  Formative 

2.  Summative 

3.  Policy  (management  information  systems)  use 


Instruaents  based  on  formal  theory  have  the  advantage  of  providing  data  in 
a  coherent  context  that  helps  organizational  consultants  and  their  clients 
interpret  results  and,  in  some  cases,  identify  specific  action  steps  to  im¬ 
prove  the  organization's  work  environment.  Theory-based  instruments  are 
■ore  likely  to  offer  supporting  literature  and  validation  data  from  other 
settings  which  increase  the  instrument's  credibility  for  users. 

The  disadvantage  of  theory-based  instruments  is  that  they  are  often 
.Strongly  biased  in  favor  of  "one  right  managerial  approach"  which  may  or 
may  not  be  relevant  to  the  organization  in  which  they  are  used.  This  is 
particularly  true  of  the  adaptation  of  civilian  instruments  for  use  in  mili¬ 
tary  settings.  Civilian  theorists'  instruments  are  frequently  based  on 
student  or  industry  samples,  and  present  data  in  a  way  that  advocates  mana¬ 
gerial  approaches  (e.g.,  "humanistic"  psychology,  "democratic-participative" 
management)  which  may  not  be  appropriate  to  military  organizations.  These 
instruments  can  have  low  face-validity  for  respondents  and  may  jeopardize 
the  credibility  of  the  intervention  effort  or  lead  consultants  to  make 
recommendations  inappropriate  to  their  clients. 

2.  Grounded  theory.  Some  instruments  were  based  on  interview,  group 
sensing,  and  observation  research  in  specific  organizations— the  "grounded" 
theory  approach  reooanended  by  Glaser  and  Strauss  (1967) .  In  this  approach, 
survey  designers  go  directly  to  organizational  members  and  ask  them  to  de¬ 
scribe  their  concerns,  what  aspects  of  their  organization  contribute  to 
satisfaction  and  productivity,  and  what  aspects  result  in  low  morale  and 
performance.  These  open-ended  responses  are  then  contint-analyzed  by  the 
survey  developers,  who  select  the  most  frequent  comments  and  write  them 

in  survey  item  form,  using  the  organization  members'  own  words  as  much  as 
possible.  The  resulting  "tailored"  survey  is  then  administered  to  the  or¬ 
ganization's  members  to  collect  quantitative  data. 

The  advantage  of  this  approach  is  that  it  results  in  an  instrument 
that  is  face-valid  and  directly  relevant  to  the  concerns  of  organizational 
ambers  being  surveyed.  This  may  increase  respondents'  motivation  to  com¬ 
plete  the  survey  accurately  and  help  organizational  consultants  win  accep¬ 
tance  for  an  intervention  program.  The  disadvantage  of  grounded  surveys 
is  that  responses  to  an  assorted  collection  of  items  may  prove  difficult 
to  interpret  or  to  feed  back  to  respondents . 

3.  empiricism.  Same  instruments  are  developed  by  administering 
lengthy  questionnaires  containing  items  drawn  from  a  variety  of  sources, 
then  computer- analyzing  responses  to  identify  items  and  factors  which  seem 
to  relate  to  the  organization's  functioning.  This  approach  has  the  advan¬ 
tage  of  examining  in  an  unbiased  way  the  widest  range  of  possible  variables 
that  My  be  important  in  a  given  setting. 

Its  disadvantages  are  that  the  resulting  survey  may  not  appear  rele¬ 
vant  to  respondents  in  the  organization  and  that  "shotgun  empiricism"  fre¬ 
quently  produces  too  many  variables  unrelated  by  coherent  structure  or 
theory,  which  makes  use  and  interpretation  of  results  difficult. 

Most  climate  instruments  involve  the  use  of  all  three  formal  theory- 
based  approaches  at  some  point  in  their  development,  instruments  usually 


origins.  For  example,  ths  ISR  Survey  of  Organizations 
of  tits  Any  and  navy's  present  VBQs,  was  based  origi- 
Bowera  «  Seashore,  1966) .  Empirical  analysis  of 
wded  . theory-baaed  instruments  often  results  la  their  modi- 
Arny  General  Organisation  Questionnaire  and  Navy  Human  Bu¬ 
rnt  Survey  have  diverged  from  their  civilian  parent  as  a 
analysis. 


ansa irises  the  theoretical  bases  of  the  instruments  reviewed 
Approximately  half  of  the  38  instruments  examined 
from  10  identifiable  theoretical  frameworks.  No  single 
dominates  the  field  as  a  source  of  instnaiants ,  although  the  Likert/ 
and  Seashore  fraamwork  has  been  most  widely  used  in  the  Army  and 
tbm  theoretical  antecedents  of  an  instrument  generally  influence 
original  pool  of  items.  For  example,  the  Likert/Bowers  and  Seashore 
interpersonal  variables,  paying  less  attention  to 


job  or  task  Variables .  By  contrast,  Hackman  and  Lawler's  frostwork  focuses 


almost  exclusively  on  job  and  task  variables. 


Another  form  of  influence  occurs  when  a  normative  theoretical  frame¬ 
work  is  used.  In  this  cess  there  is  an  increased  tendency  to  use  evalua¬ 
tive  rather  than  descriptive  items,  thereby  increasing  the  possibility  of 
blasts. 


Finally,  some  theoretical  frameworks  determine  not  only  the  pool  of 
items,  bet  also  the  form  of  the  questions,  this  is  most  readily  seen  in 
the  case  of  expectancy  theory  and  th*  Army's  mode  environment  questionnaire 
where  the  theory  requires  assessments  of  the  importance  of  particular  out- 
and  their  expected  contingencies . 


the  number  of  explicitly  grounded  theories  is  small,  this  is 
what  misleading,  however,  since  a  number  of  the  theory-based  instruments 
also  utilise  interview  end  observational  techniques  to  establish  item 
pools. 


the  empirical  category  is  somewhat  of  s  residual  category  since  it 
includes  iatnmmnte  spliced  together  from  pre-existing  instruments  (Michi¬ 
gan  Organizational  Assessment  Package) ,  instruments  based  on  particular 
groups  or  settings  (Work  Climate  of  Priests  and  Agency  climate  Question¬ 
naire),  pnd  instruments  that  give  every  indication  of  generating  formal 
theoretical  frameworks  (Ward  Atmosphere  Seale  end  Military  Company 
mvircwmmnt) . 

flw  relatively  large  matter  of  empirically-based  instruments  under¬ 
lines  the  dilemne  associated  with  choosing  either  a  general  instrument  or 
ana  specifically  tailored  to  the  setting.  The  dominance  of  the  letter 
dotation  substantially  reduces  ths  opportunities  for  cross-validation  of 
mama  the  subscales  of  pre-existing  instruments. 


Table  2 


Bases  of  Instruments  Reviewed 


1.  Formal  Theory 

a.  Likert /Bowers  £  Seashore 


b.  Ohio  Studies  in  Leadership 

c.  Fiedler-Contingency  Theory 
of  Leadership 

d.  Porter  &  Lawler's  Expectancy 
Model  of  Motivation 

e.  Mas low's  Theory  of  Needs 

f.  Blake  £  Mouton  Two  Factor 
Theory 

g.  Murray  £  Lew in 

h.  McClelland  £  Atkinson 

i.  Contingency  Theory 

j.  Hackman  £  Lawler,  Job 
Characteristics 

2.  Grounded  Theory 


Profile  of  Organizational 
Characteristics 
Survey  of  Organizations 
General  Organizational 
Questionnaire 

Human  Resource  Management  Survey 

Military  Leadership  Survey 
Organizational  Climate  for 
Schools 

Leader-Match 


Work  Environment  Questionnaire 


Alderfer's  Satisfaction  and 
Desire  Measures 

Organizational  Climate 
(Greiner  et  al.J 

Business  Organization  Climate 
Index 

College  Characteristics  Index 

Organizational  Inventory 
Organizational  Climate  Survey 
Questionnaire 

Work  Analysis  Questionnaire 
Performance  Audit 

Organizational  Description 
Questionnaire 

Job  Diagnostic  Survey 


Measure  of  Morale 
Leadership  Evaluation  and 
Analysis  Program 
Motivation  fi  Working  Relations 
of  Scientists  £  Engineers 
Questionnaire 

Measures  of  Military  Attitudes 
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Table  2  (Continued) 


3.  Empirical  Employee  Motivation  and  Morale 

Military  Company  Environment 
Inventory 

Hard  Atmosphere  Scale 
Work  Environment  Seale 
Organization  Survey  (USAREUR) 
Michigan  Organizational 
Assessment  Package 
Occupational  Attitude  Inventory 
Organization  Development  Survey 
Campbell  &  Pritchard  Instrument 
Agency  Climate  Questionnaire 
Work  Climate  of  Priests 
Survey  of  Management  Climate 


Organizational  Components  Assessed 

Theories  of  organizational  behavior  usually  propose  a  systems  model  to 
illustrate  the  components  or  factors  of  the  organization  and  their  inter¬ 
actions  (Likert,  1967,  1973;  Lawrence  &  Lorsch,  1970;  Sieber,  1973;  East  & 
Rosenzweig,  1970) .  Work  environment  questionnaires  can  be  analyzed  by 
identifying  and  classifying  items  and  variables  in  terms  of  the  components 
and  interactions  they  measure.  The  six  organization  components  or  factors 
illustrated  in  Figure  2  were  used  as  a  starting  point  in  the  present  study. 

Supervisor  Variables 

Any  item  or  variable  which  related  to  supervisor  behavior,  managerial 
practices,  or  style  was  recorded  under  this  component.  The  supervisor 
component  was  used  for  the  "superior"  or  "person  with  power”  in  analogous 
organization  situations  (e.g.,  the  teacher  in  classroom  HEQs,  doctor  or 
staff  in  hospital  settings,  and  the  warden  or  guards  in  prison  environ- 
ments) .  Example s  "My  supervisor  is  friendly  and  easy  to  approach.” 

Job  Variables 


Any  item  or  variable  that  related  to  the  work  itself  (the  physical 
setting,  technology,  design  variety,  meaningfulness  or  intrinsic  feedback 
resulting  from  task  activities  in  the  organization)  was  recorded  under 
this  component .  Examples t  "I  get  to  do  many  interesting  tasks  in  this 
job"  or  "I  don't  have  adequate  job  training.” 

Subordinates  Variables 


Any  item  or  variable  that  described  the  personalities,  motivation, 
skills,  or  other  characteristics  of  individuals  in  the  organization  or 
peers  in  work  groups  was  recorded  under  this  component.  In  analogous 
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Figure  2.  Organizational  components  in  a  causal  flow  model. 
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organizations,  "supervisees"  (persons  inferior  in  power,  such  as  students 
in  educational  settings,  patients  in  hospitals,  and  prisoners)  were  con¬ 
sidered  "subordinates."  Examples:  "Z  want  to  do  the  best  job  Z  can  in 
this  job"  or  "My  co-workers  do  not  care  about  doing  a  good  job." 

Work  Group  Relations 

__  Any  item  or  variable  which  dealt  with  working  relations  in  primary 
work  groups  (peer  leadership,  interactions  involving  face-to-face  con¬ 
tacts,  cooperation  or  conflict,  norms,  standards,  and  procedures)  was 
recorded  under  this  component.  Examples:  People  are  willing  to  help 
each  other  out  in  this  work  group"  or  "NCOS  seldom  listen  to  one  another. " 

Organization  Variables 

Items  or  variables  which  described  the  organization  as  a  whole  (its 
norms,  standards,  rules,  policies,  or  procedures)  were  recorded  under  this 
component .  Examples:  "There  is  continual  pressure  to  improve  job  per¬ 
formance  in  this  organization"  or  "Nobody  is  concerned  about  discipline 
in  this  unit." 

Exogenous  Variables 

Items  or  variables  which  dealt  with  factors  outside  the  organization, 
such  as  the  economy,  culture,  competition,  relations  with  supplier  or  cus¬ 
tomer  organizations,  union,  legal,  regulatory  or  political  influences,  were 
recorded  under  this  component.  Example:  "My  decision  to  reenlist  will 
depend  on  how  good  a  job  I  think  I  can  get  in  the  civilian  economy." 

Organization  models  often  relate  these  components  in  a  causal  flow 
sequence  which  indicates  how  component  variables  influence  one  another  to 
produce  organizational  outcomes  (Likert,  1973;  Fecorella,  Hausser  &  Wiss- 
ler,  1974;  Franklin  &  Spencer,  1974;  Turney  &  Cohen,  1976).  Directionality 
and  strength  of  path  correlation  coefficients  are  established  by  cross-lag 
correlation  designs  (see  discussion  under  "Validity"  below) .  Figure  2  sets 
out  the  six  organizational  components  used  in  the  present  study  in  a  causal 
flow  sequence.  Path  coefficients  illustrated  are  from  Likert  (1973)  ,  but 
Franklin  (1973)  reports  quite  different  directionalities  and  path  coeffi¬ 
cient  statistics  (the  Likert  model  does  not  recognize  "job"  and  "exogenous" 
variables) . 

As  indicated  in  Table  3,  the  work  environment  instruments  reviewed 
differed  considerably  in  the  organizational  components  they  assessed  and 
the  extent  to  which  the  assessed  component  was  specified.  Analysis  of 
instrument  variables  indicated  the  "subordinate"  and  "work  group”  com¬ 
ponents  tended  to  be  redundant:  Most  "subordinate"  items  and  variables 
(e.g.,  "My  co-workers  are  f riendly/cooperative /motivated. ”)  could  also  be 
classified  as  "work  group”  variables.  Consistent  with  the  decision  to 
focus  on  organizational  rather  than  individual  personality  variables, 
it  was  decided  to  record  judgments  about  peers  as  "work  group"  indices, 
and  reports  of  the  subject's  own  motivation,  morale,  or  satisfaction  as 
"satisfaction  outcomes"  rather  than  causes  (as  suggested  by  Motowidlo  et 
al. ,  1976).  "Exogenous"  and  "task  performance  outcome"  items  and  variables 
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rarely  appeared  in  the  instruments  reviewed  and  were  subsumed  by  the  or¬ 
ganization  category. 


Variables  and  items 


Reviews  of  the  work  environment  and  organizational  climate  instru¬ 
ments  usually  attempt  to  identify  generic  variables,  or  variable  factors 
that  seem  to  appear  in  all  instruments  reviewed,  and  to  describe  attributes 
common  to  all  organizations. 

Campbell  et  al.  (1970),  in  reviewing  this  literature,  identified  four 
factors  as  common  across  all  the  investigations: 

individual  autonomy;  the  degree  to  which  an  individual  feels  that 
responsibility  has  been  delegated  to  him,  that  he  can  take  initia¬ 
tive,  make  decisions  and  "be  his  own  boss"  without  having  to  be 
constantly  accountable  to  higher  management  (same  instruments 
phrase  this  variable  factor  in  negative  terms,  e.g.,  "conformity," 
"rules  of  orientation,"  or  "over-control,"  to  measure  managerial 
practices  or  organizational  procedures  which  deny  people  autonomy, 
responsibility,  authority,  or  influence  in  their  jobs); 

structure;  the  degree  to  which  objectives  and  methods  for  job  per¬ 
formance  are  established  by  the  organization  and  communicated 
clearly  to  individual  members,  and  the  extent  to  which  managers 
plan  and  organize  tasks  efficiently; 

reward  orientation ;  the  degree  to  which  individuals  in  an  organi¬ 
zation  receive  feedback  (primarily  positive),  rewards ,  and  recogni¬ 
tion  for  task  performance  or  achievement  of  objectives; 

consideration ,  warmth,  and  support:  the  degree  to  which  managers 
or  the  organization  provide  support  for  individuals,  relate  to  them 
socially,  and  help  them  with  or  take  into  consideration  their  per¬ 
sonal  needs  in  assigning  them  tasks. 

This  typology  of  work  environment  variable  factors  is  congruent  with  the 
two  major  factors,  task  emphasis  (initiating  structure)  and  people  empha¬ 
sis  (consideration)  found  in  factor  analyses  of  leadership  styles  (S tog- 
dill,  1974). 

Moos  (1974) ,  summarizing  the  literature  on  organizational  climate 
instruments  used  in  education  and  health  care  institutions,  identified 
three  basic  dimensions: 

relationship:  "The  involvement  of  individuals  in  the  environment 
and  the  extent  to  which  they  support  and  help  each  other"  (in  Moos' 
Military  Company  Environment  Inventory,  1973,  this  dimension  is 
called  "peer  cohesion") ; 

personal  development:  autonomy  ("the  extent  to  which  people  are 
encouraged  to  be  self-sufficient  and  independent") ,  the  amount  of 
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training  or  informal  coaching  individuals  receive,  and  the  empha¬ 
sis  placed  on  personal  career  planning  and  advancement ; 

systems  maintenance i  the  degree  of  order  and  organization,  clarity 
of  expectations  about  maintenance  and  task  objectives,  and  the  ex¬ 
tent  of  management  control  of  people  and  outcomes. 

Moos  observes  that  in  "work  milieus"  an  additional  "task"  factor,  variously 
called  "task  orientation"  and  "work  pressure,"  is  usually  present. 


The  present  study  identified  10  organizational  process  and  2  outcome 
factors  which  adequately  encompassed  the  items  and  variables  in  the  instru- 
ments  reviewed,  while  providing  greater  discrimination  among  variable  di¬ 
mensions  than  that  offered  by  the  Campbell  et  al.  and  Moos  typologies. 

Items  and  variables  dimsnsions  which  described  observable  specific 
behaviors  or  characteristics  of  jobs  or  the  organization  were  considered 
process  (e.g.,  "causal")  variables,  and  were  distinguished  where  possible 
from  outcome  (e.g.,  satisfaction  and  performance)  variables.  For  example, 
"This  organisation  sponsors  many  social  activities  for  its  employees”  was 
considered  a  process  variable  (an  action  of  the  organization) ,  while  "I 
like  my  co-workers”  was  considered  a  satisfaction  outcome  (a  result  of  the 
organisation's  actions).  Ambiguous  items  which  involved  evaluative  re¬ 
sponses  (e.g. ,  "Caamunications  are  good  in  my  work  group"  or  "Majority  and 
minority  personnel  work  together  well”)  were  considered  process  variables 
unless  they  required  the  respondent  to  report  subjective  feelings. 

Process  Variables 

The  10  process  variables  were: 

1.  Support  t  The  degree  to  which  supervisors  or  the  organization  pro¬ 
vide  for  the  personal  needs  or  well-being  of  individuals.  Three  subcate¬ 
gories  of  this  factor  were  distinguishable: 

e  personal  support:  a  willingness  to  listen  to  and  help  indi¬ 
viduals  with  personal  problems  and  to  take  their  needs  into 
account  in  making  personnel  decisions.  Examplet  "I  can  talk 
to  ay  boss  about  personal  problems.” 

e  organizational  services t  the  adequacy  or  inadequacy  of 

organization-provided  facilities  and  services,  such  as  recrea¬ 
tion,  housing,  transport,  and  banking,  relating  to  employees' 
personal  needs.  Example :  "This  installation  has  no  recrea¬ 
tional  facilities." 

•  hygiene t  physical  aspects  of  the  work  environment  (lighting, 
sanitation,  noise,  vibration,  dirt,  caustic  chemicals,  un¬ 
safe  equipment)  that  affect  employees'  health  or  physiological 
security.  Example :  "X  am  exposed  to  toxic  gases  on  my  job." 

2.  Personnel  Development :  (1)  the  degree  to  which  supervisors  or  the 
organization  formally  train  or  informally  coach  their  subordinates  to  improve 
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thair  knowledge,  skills,  or  job  performance;  encourage,  counsel,  or  help 
subordinates  to  advance  in  their  careers;  or  make  education  and  personal 
developamnt  opportunities  available  to  them;  (2)  the  opportunity  subordi¬ 
nates  have  to  use  thair  skills  or  training  in  their  present  jobs;  and 
(3)  the  extent  to  which  they  feel  competent  and  adequately  prepared  to 
do  the  tasks  they  are  assigned.  Examples:  "My  boss  seldom  shows  me  how 
to  iaprove  ay  job  performance."  "I  receive  adequate  training  to  be  able 
to  do  ay  jab.”  "My  job  fails  to  make  full  use  of  my  talents,  skills  and 
training.” 

3.  Teaxmtork:  the  degree  to  which  co-workers  work  well  together. 

This  variable  factor  includes  three  distinguishable  subcategories: 

•  social  interaction:  whether  co-workers  act  in  friendly,  re¬ 
spectful  ways  toward  one  another,  interact  socially  voluntarily 
within  and  outside  the  work  place,  and  maintain  amicable  racial 
and  intragroup  relations.  Examples:  "Majority  and  minority 
personnel  work  together  well  in  this  unit.”  "My  work  group 
seldom  gets  together  socially  outside  working  hours.” 

e  work  facilitation:  whether  people  cooperate  to  help  one  an¬ 
other  or  other  groups  to  accomplish  tasks,  as  opposed  to  com¬ 
peting,  sabotaging  one  another,  withholding  information  or 
otherwise  conflicting  in  jobs  which  require  group  effort. 
Example:  "My  unit  is  unwilling  to  help  other  units  when  they 
need  a  hand."  Supervisor  or  peer  conflict  resolution  activi¬ 
ties  were  also  considered  work  facilitation.  Examples:  "My 
supervisor  encourages  members  of  our  work  group  to  work  to¬ 
gether  as  a  team."  "Conflicts  in  this  organization  are  openly 
dealt  with  and  resolved." 

e  group  cohesiveness :  whether  supervisors  or  work  group  members 
act  to  create  feelings  of  group  identity,  such  as  through  the 
development  of  a  symbol  (e.g. ,  insignia,  mascots,  berets)  for 
the  group,  or  competing  with  other  groups  in  ways  that  develop 
team  spirit  without  detracting  from  organizational  performance 
(e.g.,  athletics).  Example:  "We  have  a  unit  insignia  that 
sets  us  apart  from  other  units." 

4.  Conmmicatlons  Clarity:  the  degree  to  which  people  feel  they  are 
"in  the  know";  that  supervisors'  rules,  standards  or  procedures  and  the 
organization's  goals  are  clear  and  understandable;  that  communication  chan¬ 
nels  are  open  up  and  down  the  chain  of  command;  and  that  effective  hori¬ 
zontal  and  vertical  communication  takes  place.  Examples:  "I  get  the  in¬ 
formation  X  need  to  be  able  to  do  my  job."  "X  know  exactly  what  is  expected 
of  me  in  my  job."  "Communications  are  poor  between  my  unit  and  other  work 
groups  in  this  organization."  "My  boss  has  an  open  door  policy." 

5.  Order  and  Organization:  the  degree  to  which  individuals  perceive 
their  work  group  or  agency  to  be  well  organized,  the  task  planning  and  co¬ 
ordination  efficient  and  effective,  and  the  logistic  support  (e.g.,  neces¬ 
sary  equipment,  resources,  and  support  services)  timely  and  adequate. 
Examples:  "My  work  group  is  poorly  organized.”  "The  missions  this  company 
undertakes  are  carefully  planned  so  nothing  goes  wrong."  "X  seldom  get  the 
tools  and  equipment  X  need  to  do  my  job." 
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6.  Standards:  the  degree  to  which  supervisors  or  the  organization 
express  concern  with  or  set  high  standards,  challenging  specific  objectives 
for  task  performance,  personal  conduct,  or  the  maintenance  of  physical 
facilities  and  equipment.  Examples:  My  supervisor  sets  high  standards 
for  work  done  in  my  unit."  "There  is  continual  pressure  to  improve  per¬ 
formance  in  this  organization."  "I  have  specific  objectives  for  my  per¬ 
formance  in  my  job  this  year." 

7.  Job  Design:  the  degree  to  which  the  job  itself  seems  relevant 
or  important,  or  provides  meaningful  activities,  challenges,  variety, 
identity  with  a  completed  whole,  or  intrinsic  feedback  or  rewards  to  indi¬ 
viduals  on  how  well  they  are  doing  (cf.  Hackman  &  Oldham,  1974).  Ex¬ 
amples:  "We  spend  a  lot  of  time  sitting  around  doing  nothing,  without 
any  meaningful  work."  "I  get  to  do  many  different  tasks  in  this  job." 

"My  job  gives  me  something  to  show  for  my  efforts  when  my  work  is  done." 

"My  job  allows  me  to  set  my  own  pace." 

8.  Responsibility  (Autonomy  and  Influence) :  the  degree  to  which 
people  feel  they  can  make  job-related  decisions  without  having  to  get  the 
approval  of  superiors,  are  in  control  of  their  own  situations,  and  have 
influence  within  their  work  groups  or  the  organization,  especially  with 
those  of  higher  or  equivalent  status.  "Over  control"  and  "conformity" 
items  concerning  organizational  aspects  which  limit  members'  responsibility 
were  also  classified  for  this  factor.  Examples:  "I  can  plan  my  work 
without  having  to  check  with  my  boss."  "This  organization  has  a  lot  of 
Mickey  Mouse  rules  and  red  tape  which  get  in  the  way  of  getting  the  job 
done."  "I  can  influence  decisions  that  affect  my  working  conditions." 

9.  Feedback  and  Rewards:  the  degree  to  which  individuals  get  feed¬ 
back  from  their  supervisors,  co-workers,  or  organization  on  how  well  they 
are  doing  their  jobs,  and  the  extent  to  which  they  feel  good  task  per¬ 
formance  is  rewarded  (e.g. ,  by  praise,  promotions,  compensation,  favors, 
time  off,  special  awards,  and  commendations).  Examples:  "My  supervisor 
seldom  gives  me  feedback  on  how  well  I  am  doing  my  job."  "People  who  do 
a  good  job  get  rewarded  in  this  organization."  "If  someone  doesn't  do 
his  job,  this  work  group  lets  him  hear  about  it." 

10.  Justice:  the  degree  to  which  people  in  the  organization  feel 

job  assignments,  training  opportunities,  promotions,  and  other  rewards, 
and  disciplinary  actions  and  punishments  are  handled  on  a  fair  and  equal 
basis,  without  favoritism  or  discrimination  because  of  race,  sex,  or  other 
individual  characteristics.  Examples:  minorities  get  the  same  oppor¬ 

tunities  for  advancement  as  majority  personnel  in  this  organization." 
"Disciplinary  actions  are  not  handled  fairly  here."  "My  supervisor  does 
not  'play  favorites'  in  deciding  who  gets  which  job." 

Outcome  Variables 


Items  and  variables  relating  to  the  two  outcome  variable  categories, 
task  performance  and  satisfaction,  were  also  identified  in  many  instruments. 

1.  Performance:  the  degree  to  which  the  organization  achieves  pro¬ 
ductivity,  efficiency,  goal  or  mission  accomplishment  outcomes  as  a  result 
of  managerial  job  design,  peer  interaction,  or  organizational  practices. 


31 


This 


Examples:  "My  unit  usually  accomplishes  its  mission  objectives." 
company ' s  score  on  the  last  ARTEP  inspection  was  _ . " 

2.  Satisfaction:  the  degree  to  which  members  feel  positive  about 
themselves,  their  organization,  supervisors,  job,  co-workers  (affective 
evaluations  of  motivation,  satisfaction,  or  morale) .  Satisfaction  may 
also  be  evidenced  indirectly  in  measurable  organizational  outcomes  such 
as  turnover  or  retention  rates,  grievances,  health,  or  other  results  re¬ 
lated  to  feelings  of  satisfaction.  Examples:  "1  like  my  job."  "Every¬ 
thing  considered,  military  service  offers  me  a  good  career.”  "I  do  not 
intend  to  reenlist. "  "First  term  retention  rates  in  this  company  were 
_  percent." 


Table  3  summarizes  process  and  outcome  variables  and  organization 
components  found  in  the  instruments  reviewed. 

Content  Analysis  of  Instruments 

Thirty-one  of  the  38  instruments  containing  over  4,000  items  were  re¬ 
viewed  in  am  item-by-item  content  analysis  (seven  instruments  were  dropped 
because  of  insufficient  or  partial  data) .  This  approach  was  chosen  be¬ 
cause  an  initial  analysis  of  factors  and  scales  proved  inadequate.  Factors 
frequently  appeared  to  be  misnamed  when  the  individual  items  were  examined; 
the  use  of  varimax  rotation  procedures  in  the  factor  analysis  tended  to 
limit  the  number  of  factors  generated  (Stern,  1970) ,  without  adding  any¬ 
thing  to  their  comprehensibility.  A  priori  scales  covered  such  a  wide 
range  of  variables  and  nomenclatures  that  direct  comparisons  across  instru¬ 
ments  proved  impossible. 

Item-by-item  analysis  also  permitted  an  analysis  of  the  instruments 
by  the  organizational  component  involved.  The  first  attempt  at  an  item- 
by-item  analysis  was  guided  by  the  most  frequently  mentioned  factors  and 
subscales.  These  were  expanded  or  contracted  as  a  result  of  trying  to  al¬ 
locate  individual  items.  A  fairly  stable  set  of  23  variables  emerged,  20 
of  which  were  process  variables  and  3  of  which  were  outcome  variables. 
Twenty-six  instruments  were  analyzed  using  this  framework.  A  smaller  frame¬ 
work  containing  the  10  process  and  2  outcome  variables  discussed  above  was 
then  derived  from  the  larger  framework  by  grouping  redundant  variables. 

While  the  reduction  in  the  number  of  variables  decreased  the  discriminat¬ 
ing  power  of  the  content  analysis  of  the  instruments,  it  produced  a  more 
parsimonious  framework.  All  instruments  reviewed  were  recoded  using  this 
variable  framework. 

As  can  be  seen  from  Table  3,  only  the  presence  or  absence  of  a  variable 
is  indicated.  This  provides  some  control  over  the  number  of  items  in  a 
questionnaire:  Longer  instruments  dominate  only  to  the  extent  to  which 
they  have  a  broader  coverage.  The  Table  3  summaries  of  the  individual  in¬ 
struments  provide  a  coarse-grained  analysis  permitting  visual  comparison 
of  ccoponents  and  variable  dimensions  of  instruments  reviewed.  Some  con¬ 
fusion  in  assigning  items  to  particular  cells  was  unavoidable,  due  to  the 
overlap  of  variables  and  the  multi-dimensionality  and  ambiguity  of  many 
items. 
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Classifications  are  not  mutually  exclusive.  A  statement  such  as,  "I 
can  talk  to  my  boss  about  decisions  affecting  my  job,"  can  be  classified 
under  both  Support  and  Communications  Clarity.  Similarly,  an  item  can  be 
classified  under  more  than  one  organizational  component  when  it  is  unclear 
whether  the  item  refers  to  the  respondent's  job,  work  group,  or  organiza¬ 
tion.  In  cases  where  the  actual  survey  instrument  was  not  available, 
judgments  had  to  be  made  from  partial  reports  in  published  descriptions; 
in  such  instances,  the  names  of  variable  scales  or  factors  reported  were 
used  rather  than  the  actual  items. 


Instrument  by  Organizational  Component.  Military  instruments  tended 
to  focus  on  the  role  of  the  superior  in  determining  a  unit's  organizational 
climate,  with  markedly  fewer  items  concerned  with  peer  group  and  subordi¬ 


nate  roles  in  determining  organizational  climate.  Job-related  questions 


also  appear  to  be  relatively  underrepresented. 


The  same  general  distribution  of  items  also  exists  for  civilian  in¬ 
struments,  but  the  emphasis  on  the  supervisor  component  is  less  pronounced. 


Instrument  by  Process  Variable.  Military  instruments  emphasized  Sup¬ 
port,  Personnel  Development,  and  Feedback  and  Reward  variables,  and  con¬ 
tained  decidedly  fewer  items  relating  to  Job  Design,  Communication  Clarity, 
Order  and  Organization,  and  Justice  variables. 


Civilian  instruments  focused  more  on  Responsibility  and  Teamwork 
variables,  and  contained  fewer  items  related  to  Communication  Clarity,  Job 
Design,  and  Justice.  The  consistent  underemphasis  of  Communication  Clarity 
may  in  part  be  an  artifact  of  the  way  the  variables  were  constructed:  Com¬ 
munication  as  a  process  was  divided  between  Performance  Feedback,  Work 
Facilitation,  and  general  Communication  Clarity.  The  underemphasis  of  the 
Order  and  Organization  variable  is  less  readily  interpretable. 

The  overall  pattern  for  the  military  instruments  suggests  a  focus  on 
bureaucratic  organizations  emphasizing  support,  training,  and  discipline, 
whereas  civilian  instruments  put  more  stress  on  Teamwork  and  individual 
Responsibility  variables. 

The  single  greatest  difference  between  military  and  civilian  instru¬ 
ments  is  in  the  frequency  of  the  Justice  variable .  Its  predominance  in 
the  military  instruments  is  probably  due  tc  the  racial  composition  of  the 
sdlitary  services  and  the  amount  of  discretionary  authority  possessed  by 
NCOS  and  officers  with  respect  to  lower  level  enlisted  men. 


Table  4  summarizes  the  reliability  statistics  reported  for  the  in¬ 
struments  reviewed.  Most  of  the  instruments  were  tested  across  sizable 
populations ,  with  the  military  instruments  in  particular  having  large 
data  bases.  As  the  table  indicates,  only  15  out  of  37  instruments  pro¬ 
vide  statistical  information  on  the  internal  consistency  reliabilities. 
Where  several  variable  scale  reliabilities  or  a  range  of  reliabilities 
attained  with  different  administrations  of  the  instrument  were  reported, 
the  figures  given  in  Table  4  are  the  mean  scores.  It  can  be  seen  that 
reliabilities  for  work  environment  and  organizational  climate  question¬ 
naires  range  from  .24  to  .99,  with  a  mean  of  .72  for  scale  reliabilities 
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and  .69  for  test-retest  reliabilities.  Reliability  of  .70  is  probably  a 
realistic  standard  for  instruments  of  this  type. 

Retest  reliabilities  were  only  available  for  6  instruments.  Of 
these (  only  one  instrument,  Patchen's  Employee  Motivation  and  Morale  Test, 
had  a  test-retest  reliability  at  a  highly  acceptable  level  (r  -  .80) ,  al¬ 
though  the  remaining  five  had  reliabilities  in  excess  of  .60. 

Some  20  out  of  37  instruments  used  some  form  of  factor  analysis  to 
establish  subscales  or  dimensions.  Factor  analysis  is  essentially  a 
statistical  technique  for  clustering  items  and  variables  which  correlate 
with  one  another.  It  can  be  used  to  identify  or  confirm  the  existence  of 
variable  scales  in  a  large  number  of  unrelated  items,  summarize  data  par¬ 
simoniously,  and  can  assure  that  such  scales  have  high  internal  consis¬ 
tency  reliability  but  the  method  has  a  number  of  methodological  limita¬ 
tions.  First,  items  which  do  not  sensibly  relate  to  one  another  may 
statistically  factor  together,  causing  problems  of  factor  interpretation 
and  labeling  (e.g. ,  items  relating  to  discipline,  supervisor  support, 
standards  and  respondent  rank,  leading  to  a  "lower-enlisted-discipline- 
standards- support"  factor) .  Factors  of  this  type  cannot  be  meaningfully 
interpreted  and  have  little  utility  as  a  basis  for  organizational  devel¬ 
opment  interventions.  Second,  to  be  valid,  factor  analyses  require  large 
numbers  of  items  (30  items  per  variable)  and  luge  samples  (10  subjects 
per  item  or,  for  example,  1,800  subjects  for  a  six-variable  instrument 
(Nunnally,  1967)).  In  general,  factor  analysis  is  best  used  to  test  the 
existence  of  hypothesized  factors  rather  than  to  develop  instruments,  as 
in  "shotgun  empiricism"  approaches.  Factor  reliabilities  are  reported 
in  terms  of  scale  reliability  correlation  coefficients.  Table  4  summar¬ 
izes  the  factor  data  available  for  instruments  reviewed  in  the  present 
study.  Table  5  sumnarizes  this  data  by  work  environment  variable  and 
performance  and  satisfaction  outcomes.  While  some  studies  report  other 
statistical  tests  (t  statistics,  analysis  of  variance,  and  Chi-square  or 
other  nonpar ame trie  analyses) ,  only  correlation  coefficients  (absolute 
values)  are  reported  because  they  provide  the  only  comparable  statistics 
reported  for  all  instruments.  The  overall  mean  of  these  statistics  pro¬ 
vides  an  estimate  of  the  state  of  the  art  in  this  field  (following  the 
"meta-analysis"  assumptions  advanced  by  Glass,  1976,  and  Light  and  Smith, 
1971,  for  accumulating  evidence  from  a  number  of  studies  in  a  field  despite 
differences  in  their  methodologies  and  dependent  variables) .  Where  a 
range  of  £  values  for  a  given  instrument  or  variable  were  reported,  the 
m ean2  was- used .  Some  studies  reported  only  significant  correlation  find¬ 
ings,  so  the  figures  in  Table  5  probably  represent  the  mean  maximum  con¬ 
current  and  predictive  validity  levels  achievable  with  these  instruments. 
Again,  it  should  be  cautioned  that  the  findings  summarized  are  based  on 
available  data  only;  other  findings  may  exist. 

Table  5  also  indicates  these  variable  dimensions  which  are  most  likely 
to  relate  to  significant  organizational  outcomes,  and  by  implication  should 

2 

Mean  r  values  were  calculated  by  summing  the  squares  of  all  r  values 
reported  (Ir2) ,  finding  the  mean  of  these  squared  values  (Ir2/n)  and  then 
taking  the  square  root  of  this  value  (v'E  2/n) . 
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receive  highest  priority  in  the  design  of  work  environment  questionnaires 
and  in  organizational  effectiveness  interventions.  The  most  important 
dimensions  in  military  organizations  appear  to  be: 


Mean  r_  with 

Mean  £  with 

Total 

performance 

satisfaction 

mean 

outcomes 

outcomes 

_JL_ 

1. 

Teamwork 

.33 

.36 

.35 

2. 

Support 

.33 

.29 

.31 

3. 

Order  and  Organization 

.30 

.32 

.31 

4. 

Communications  Clarity 

.18 

.38 

.30 

5. 

Feedback  Rewards 

.26 

.33 

.30 

Justice  variables  (r  *  .39  with  satisfaction  outcomes) ,  on  the  basis  of 
one  study,  may  also  be  an  important  dimension  in  military  organizations. 

By  contrast,  the  most  important  variable  dimensions  in  civilian  or¬ 
ganizations  in  predicting  organizational  performance  and  satisfaction  out¬ 
comes  are: 


Mean  £  with 

Mean  £  with 

Total 

performance 

satisfaction 

mean 

outcomes 

outcomes 

X 

1. 

Communications  Clarity 

.50 

_ 

.50 

2. 

Standards 

.42 

— 

.42 

3. 

Teamwork 

.39 

.41 

.40 

4. 

Job  Design 

.31 

.36 

.34 

5. 

Feedback  Rewards 

.33 

» — 

.33 

Military  organizations'  results  appear  somewhat  more  dependent  on 
structural  and  support  variables,  while  civilian  organizational  outcomes 
are  more  affected  by  standards  and  job  design  variables.  Teamwork,  com¬ 
munications,  and  feedback  recognition  are  critical  to  organizational  ef¬ 
fectiveness  in  both  types  of  organizations.  Satisfaction  outcomes  are 
marginally  better  predicted  (by  organizational  variables)  than  performance 
outcomes.  Prediction  of  performance  dimensions  is  significantly  better 
in  civilian  settings  (y£  ■  .38)  than  in  military  organizations  (u£  ■  .23), 
perhaps  because  performance  outcome  variables  in  civilian  organizations 
(e.g.,  sales,  profits,  and  earnings)  are  better  defined  and  have  been 
studied  more  intensively.  Conversely,  prediction  of  satisfaction  outcomes 
is  better  in  military  settings  (y£  ■  .34)  than  in  civilian  organizations 
(v»£  •  .27),  perhaps  for  the  same  reason:  Military  organizations  find  it 
easier  to  measure  satisfaction  indices  (retention  rates  and  disciplinary 
actions)  and  place  more  emphasis  on  these  outcomes. 

Zn  balance,  the  work  environment  and  organizational  climate  status 
of  validity  studies  may  not  be  as  gloomy  as  much  of  the  literature  on 
these  measures  maintains. 

The  mean  level  of  validity  reporting  these  statistics  for  all  studies 
is  .33  (10  percent  of  the  observed  variance)  identical  to  the  maximum 
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An  explanation  of  columns  a,  b,  and  c  is  given  at  the  end  of  the  table 
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Table  5  (Continued) 
References  to  Instruments 


Mumford  (1976) 

Navy  Human  Resource  Management  Survey 

Borman  (1975) 

Army  Measure  of  Morale 

Crawford  £  Thomas  (1975) 

Navy  Human  Resource  Management  Survey 

Pels  £  Andrews  (1976) 

Motivation  and  Working  Relations  of  Scientists  £  Engineers 
Questionnaire 

Becklean  fi  Kinkaed  (1968) 

McBer  Organizational  Inventory 

Affourtit  (1977) 

Marine  Corps  LEAP 

Patchen  (1975) 

Employee  Motivation  and  Morale  Instrument 
Moos  (1973) 

Ward  Atmosphere  Questionnaire 
Gould  (1976) 

Air  Force  Occupational  Attitude  Survey 
Moos  (1973) 

Military  Company  Environment  Inventory 

Turney  a  Cohen  (1976) 

Army  Work  Environment  Questionnaire 

Klienman  (1976) 

Navy  Human  Resource  Management  Survey 

Drexler  (August  1973) 

Navy  Human  Resource  Management  Survey 

Drexler  (May  1973) 

Navy  Human  Resource  Management  Survey 

Drexler  (September  1973) 

Navy  Human  Resource  Management  Survey 

Klemp  (1975) 

Litwin-Stringer  (McBer)  Organization  Climate  Survey  Questionnaire 
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Table  5  (Continued) 

J.5.  Licherman,  Yalom,  &  Miles 

Encounter  Group  Leader  Behavior  Checklist 

16.  Taylor  s  Bowers  (1973) 

Survey  of  Organizations 


Smeary*  Table  7  statistics 


performance 


1.  Support 

2.  Personnel  Development 

3 .  Teamwork 

4.  Communications  Clarity 

5.  Order  and  Organization 

6.  Standards 

7.  Job  Design 

8.  Responsibility 

9.  Feedback  Rewards 
10.  Justice 


Military  Nonmilitary  Total 


.33 

.39 

.36 

.24 

.24+ 

.33 

.39 

.36 

.18 

.50 

.37 

.30 

.40 

.35 

.21 

.42 

.33 

.11 

.31 

.23 

.20 

.38 

.23 

.26 

.33 

.30 

— 

— 

— + 

.23 

.38 

.31 

Satisfaction 


1.  support 

2.  personnel  Development 

3.  Teamwork 

4.  Communications  Clarity 

5.  Order  and  Organization 

6.  standards 

7.  Jab  Design 

8.  Responsibility 

9.  Feedback  Rewards 

10.  Justice 


29 

.12 

.22 

21 

.28 

.24 

36 

.41 

.35 

38 

— ■ + 

.38+ 

32 

.17 

.27 

34 

— ■ + 

.34+ 

22 

.36 

.30 

46 

.17 

.35 

33 

— + 

.33+ 

39 

— + 

.39+ 

.34 

.27 

.32 

t  absent  data  not  averaged 


Table  5  (Continued) 


Total  (pr  Performance  fi  Satisfaction 


1. 

Support 

.31 

.29 

.30 

2. 

Personnel  Development 

.21+ 

.26 

.24 

3. 

Teamwork 

.35 

.40 

.38 

4. 

Communications  Clarity 

.30 

.50+ 

.41+ 

5. 

Order  and  Organization 

.31 

.31 

.31 

6. 

Standards 

.28 

.42+ 

.36+ 

7. 

Job  Design 

.17 

.34 

.27 

8. 

Responsibility 

.29 

.29 

.29 

9. 

Feedback  Rewards 

.30 

.33+ 

.32+ 

10. 

Justice 

.39+ 

— + 

.39+ 

.30 

.36 

.33 

t  absent  data  not  averaged. 
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criterion  validity  r  of  .33  (10  percent  of  observed  variance)  reported  for 
the  much  better  developed  field  of  psychometric  testing  (Ghiselli,  1966). 3 
these  predictive  validity  statistics  for  work  environment  variables  exceed 
those  found  for  pure  satisfaction  instruments  (yr  -  .14;  Vrocm,  1964)  and 
provide  support  for  the  organisational  climate  construct. 


XX.  CRITERIA  FOR  SELECTION/DEVELOPMENT  OF  A  WORK 
ENVIRONMENT  QUESTIONNAIRE 


Intervention  Utility 


Work  environment  and  organization  surveys  are  used  in  three  ways: 
basic  research,  organization  development  interventions,  and  management  in¬ 
formation  and  control  system  inputs  to  top  management  for  policy  evaluation 
decisions.  Most  of  the  instruments  reviewed  in  this  study  ware  designed 
and  primarily  used  for  research.  Most  of  the  preceding  discussion  has  con¬ 
cerned  their  research  properties.  A  number  of  additional  considerations 
are  important  if  instruments  will  be  used  in  organization  development  and 
evaluation  applications.  These  considerations  will  be  discussed  in  greater 
detail  in  the  final  report  of  this  project  (an  implementation  manual  for 
work  environment  questionnaires) ,  but  where  they  bear  on  instrument  design 
or  review  criteria,  they  will  be  treated  briefly  here.  Intervention  utility 
considerations  can  be  organized  in  terms  of  administration,  processing, 
feedback,  and  evaluation . 


Administration 

Three  considerations  of  importance  here  are  instrument  length  and 
administration  time,  administration  instructions,  and  resources  and  facili¬ 
ties  required. 


Instrument  Length  and  Administration .  As  discussed  above,  100  items 
and  1  hour  appear  to  be  the  outside  limits  for  instruments  which  will  be 
administered  to  large  samples  in  practical  applications.  (Many  consultants 
consider  80-item  questionnaires  too  long  and  would  prefer  surveys  of  20  to 
40  items.)  Surveys  requiring  more  than  an  hour  risk  involving  fatigue  fac¬ 
tors  (which  may  cause  instrument  rejection  responses)  and  organizational 
resistance  if  people  are  taken  away  from  their  regular  tasks  for  too  long 
a  time.  Most  of  the  instruments  reviewed  would  be  considered  too  long  by 
this  criterion. 


Administration  Instructions.  These  appear  to  be  critical  in  gaining 
instrument  acceptance,  which  in  turn  is  often  critical  to  the  success  of 


Naan  multiple  r  criterion  validities  of  .40,  -.50,  and  as  high  as  .87 
are  reported  for  the  composite  scores  of  some  instruments,  hut  as  multiple 
£  statistics  take  maximum  advantage  of  chance  and  these  findings  do  not  ap¬ 
pear  to  have  been  cross-validated,  they  should  be  treated  with  some 
skepticism. 
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the  organizational  development  intervention.  Some  of  the  instruments  re¬ 
viewed  included  administration  manuals  with  procedures  for  gaining  rapport 
with  subjects,  motivating  them  to  want  to  take  the  survey  by  explaining 
its  purposes  in  terms  of  their  interests,  and  reassuring  them  about  confi¬ 
dentiality  and  other  possibly  threatening  aspects.  Administration  instruc¬ 
tions  must  also  present  clear  and  understandable  "how  to  do  it"  mechanics 
of  recording  demographic  data,  reading  items,  marking  responses  or  trans¬ 
ferring  data  to  computer-readable  scoring  sheets,  and  returning  data  and 
materials  to  the  administrators.  Most  of  the  instruments  reviewed  did  not 
include  administration  instructions,  or  provided  only  very  brief  (and  fre¬ 
quently  inadequate)  instructions  on  the  instrument  cover  page. 

Resources  and  Facilities.  These  should  be  minimized.  Consultants  em¬ 
phasize  that  the  greater  the  amount  of  materials  and  logistics  required 
(pencils,  survey  item  booklet,  scoring  sheet,  ballot  box  or  envelopes  for 
collecting  completed  instruments) ,  the  greater  is  the  chance  that  something 
will  go  wrong.  Proctored  surveys  greatly  increase  return  rates  and  reduce 
rejection  rates,  so  scheduling  large  roams  and  getting  instructions  out  to 
subjects  to  ensure  they  show  up  at  the  right  time  and  place  to  take  the 
survey  seem  unavoidable.  Instrument  sources  in  general  had  little  to  say 
about  administration  resource  and  material  requirements. 


Processing 


Considerations  here  include  scoring  or  data  input  methods,  data  analy¬ 
sis  methods  and  procedures,  resource^  required,  and  turnaround  times. 

Scoring.  Many  work  environment  questionnaires  appear  designed  to  be 
hand-scored,  an  unsatisfactorily  slow  and  cumbersome  method  for  surveys  of 
any  length  with  more  than  a  few  subjects.  Short,  simple  instruments  de¬ 
signed  to  be  self-scored  by  participants  are  practical,  however,  when  ag¬ 
gregation  and  processing  of  data  on  a  group  basis  is  not  needed.  Data  in 
formats  which  can  be  easily  transferred  to  computer-readable  punched  cards 
or  entered  directly  from  computer  terminals  is  somewhat  better.  It  is  best 
to  have  subjects  record  responses  on  computer-readable  score  sheets  that 
can  be  read  directly  into  the  data  processing  unit  with  optical  scanning 
equipment.  The  usual  advice  given  to  instrument  designers  is  to  consider 
how  data  collected  will  be  scored  and  analyzed  from  the  very  beginning  of 
the  design  process,  and  develop  an  instrument  that  can  be  scored  in  the 
fastest  and  most  efficient  way. 


Analysis .  As  with  scoring,  many  instruments  reviewed  seemed  designed 
to  be  analyzed  by  hand,  an  alternative  no  more  satisfactory  for  this  func¬ 
tion  than  for  reporting  data.  Computer  programs  providing  at  least  summary 
statistics  should  be  designed  for  instrument  analysis  concomitantly  with 
the  design  of  the  instrument  itself.  Several  computer  packages  applicable 
to  a  variety  of  instrument  formats  are  now  available  for  this  purpose  (e.g., 
the  Army's  "OE  Survey  Program,"  Mikols,  1977). 


Resources  required.  Computer  recording  and  analysis  of  data  require 
the  availability  of  optical  scanning  or  card  reading  equipment  and  a  central 
processing  unit  with  sufficient  capacity  and  flexibility  to  use  the  analysis 
program.  Organizations  typically  handle  survey  data  analysis  in  one  of 
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thrM  wtysi  (1)  by  processing  data  on  site,  using  their  own  equipment; 

(2)  by  processing  data  on  the  organisation's  main  computer  when  time  is 
available;  and  (3)  by  lulling  data  to  central  processing  facilities  main¬ 
tained  by  the  organisation  or  by  contractors. 

Few  organisations  have  the  resources  to  use  the  first  alternative. 

The  Air  Force  experimented  with  having  subjects  record  survey  responses 
on  email  cards  in  electrically  conductive  ink  which  could  be  read  and  pro- 
cesaed  immediately  using  equipment  manufactured  by  Monroe,  but  for  a  vari¬ 
ety  of  technical  reasons,  this  method  did  not  work.  The  Army  uses  a 
standard  analysis  program  which  can  be  run  on  management  information  sys¬ 
tem  office  (MXSO)  computers  available  at  most  Army  installations.  The 
Hsvy  has  all  survey  data  sent  to  two  central  processing  facilities  in 
Norfolk,  Virginia,  and  San  Diego,  California.  Most  industrial  organiza¬ 
tions  contract  for  data  processing  with  external  consulting  firms  or  use 
their  own  central  computing  facilities. 

The  Army's  system  appears  to  be  the  best  because  it  uses  the  organi¬ 
zation's  own  equipment  and  minimized  turnaround  time.  It  follows  that  de¬ 
velopers  of  work  environment  surveys  should  design  instrument  analysis 
routines  to  be  compatible  with  their  organizations'  existing  data  process¬ 
ing  equipment  where  possible. 

Turnaround  time.  An  important  factor  in  considering  data  analysis  al¬ 
ternatives  is  the  turnaround  time  that  elapses  from  the  time  a  subject 
completes  a  work  environment  questionnaire  to  the  time  he  receives  his  feed¬ 
back.  One  month  is  considered  the  outside  limit  in  survey-guided  develop¬ 
ment  interventions .  if  more  time  elapses ,  the  data  gets  stale,  particularly 
in  military  organisations  where  frequent  personnel  rotations  can  cause  sig¬ 
nificant  organisational  changes  in  short  periods  of  time.  The  use  of  cen¬ 
tral  processing  facilities  often  results  in  long  turnaround  times  (survey 
data  is  delayed  or  lost  in  the  mails)  and  should  be  avoided  where  possible 
for  this  reason. 

Feedback 

Feedback  creates  motivation  and  direction  for  change.  The  degree  to 
which  it  stimulates  organizational  change  depends  on  the  type  and  format 
of  data  feedback  as  well  as  consultant  skill  and  climate  receptivity  to 
change.  Data  considerations  include  the  number  of  variables,  format,  sali¬ 
ence,  and  consultant  and  client  training. 

Number  of  Variables.  Most  studies  of  human  cognition  and  problem  solv¬ 
ing  report  that  human  beings  cannot  reason  effectively  about  more  than  ap¬ 
proximately  seven  variables  at  any  one  time  (the  "magical  number  7  plus  or 
minus  2,”  Miller,  1956) ,  no  matter  how  complex  the  problem.  Newell  and 
Simon  (1972)  observe  that  individuals  usually  consider  less  than  five  as¬ 
pects  of  a  problem  and  that  abstractions  from  "reality”  characteristically 
involve  the  comparison  of  perhaps  two  symbolic  representations  at  a  given 
time  (p.  795) .  This  would  suggest  that  the  upper  limit  on  the  number  of 
climate  variables  useful  to  give  a  client  is  about  10.  The  Army's  General 
Organisational  Questionnaire  and  the  Navy's  Human  Resource  Management  Ques¬ 
tionnaire  now  present  consultants  and  clients  with  an  inch-thick  survey 
printout  containing  some  10,000  statistics.  Most  military  consultants  and 
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client*  consider  this  format  and  amount  of  data  entirely  too  much.  Work 
environment  questionnaire  feedback  should  include  data  reduction  routines 
to  limit  variables  to  a  manageable  number:  This  objective  should  be  a 
prime  design  criterion  for  instruments  in  the  development  phase. 

Format.  Feedback  formats  should  be  simple,  visually  impactful, 
present  a  comparison  between  an  actual  and  desired  condition,  and  indi¬ 
cate  specific  directions  for  change.  The  latter  requirements  come  from 
self-directed  change  research  (Kolb  &  Boyatsis,  1970) ,  which  indicates  that 
people  and  organisations  are  motivated  to  change  when  they  personally  ex¬ 
perience  a  discrepancy  or  dissonance  between  an  existing  (actual)  or  de¬ 
sired  (ideal)  condition.  This  felt  distance  between  actual  and  ideal  can 
be  conceptualised  as  the  amount  of  energy  or  motivation  for  change.  Dif¬ 
ferences  too  small  and  too  large  are  less  motivating  than  moderate  dis¬ 
crepancies.  Effective  work  environment  instruments  provide  feedback  data 
in  a  format  which  makes  client  recipients  personally  feel  an  actual-ideal 
discrepancy  which  motivates  them  to  constructive  action  and  which  provides 
them  with  some  direction  (i.e.,  suggests  specific  action  steps  they  can 
take) .  In  simplest  terms ,  this  change  equation  can  be  summarized  as : 

Change  results  when  "A  +  B  +  C  S  X,"  where: 

A  -  dissatisfaction  with  an  existing  condition  (i.e.,  an  "actual" 

WE  variable  score) 

B  -  clarity  about  a  desired  condition  or  abjective  (i.e.,  "ideal" 

MB  variable  score)  <  I 

C  -  a  way  to  start:  at  least  one  specific  action  step  to  begin  to 
close  the  actual-ideal  discrepancy 

X  ■  cost  of  change  (in  terms  of  time,  money,  personal  risk,  etc.) 

If  any  of  these  elements  is  neglected  in  the  change  process  or  if  the  cost 
of  change  is  more  than  the  change  is  worth,  change  is  unlikely  to  occur. 

It  folltars  that  the  most  effective  format  for  work  environment  data  is 
one  which  includes  both  actual  and  ideal  scores  for  each  variable  and  stakes 
the  motivating  discrepancy  or  direction  for  change  visually  clear.  Figure  3 
illustrates  a  feedback  format  which  meets  these  criteria.  This  feedback 
format  provides  consultants  and  clients  with  12  data  units  (rather  than 
thousands)  in  the  form  of  a  two-point,  compound  comparison  of  actual  versus 
ideal  scores  on  each  variable  which  direct  recipients'  attention  to  poten¬ 
tially  motivating  discrepancies. 

Little  information  was  available  on  the  feedback  formats  used  with  most 
instruments  reviewed)  as  noted,  many  instruments  had  been  used  for  research 
rather  than  interventions.  Most  feedback  formats  appeared  to  be  tabular 
summaries  of  statistics  or  raw  data  in  computer  printout  form.  Feedback 
forms  for  work  environment  questionnaires  to  be  used  in  organizational  in¬ 
terventions  should  emphasize  simple  visual  formats  which  provide  change 
direction  as  well  as  status  data. 
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Salience.  Relevance  and  face  validity  concerns  are,  if  anything,  more 
important  in  designing  data  feedback  format  than  in  constructing  the  data 
collection  instrument  itself.  Recipients  must  feel  the  information  they 
receive  about  themselves,  their  subordinates,  or  their  organization  is  real, 
believable,  and  of  some  importance  to  them.  Relevance  is  helped  by  provid¬ 
ing  clients  with  criterion  validity  data  that  enables  consultants  to  show 
direct  links  between  survey  variables  and  military  unit  outcomes,  and  per¬ 
formance  results  as  well  as  satisfaction  indicators.  Face  validity  is  aided 
by  providing  specific  examples  ("war  story"  anecdotes) ,  using  military  be¬ 
havioral  science  versus  language  in  naming  variables  to  illustrate  the  con¬ 
crete  meaning  of  each  variable  dimension,  and  avoiding  normative  biases  and 
advocacy  of  a  particular  managerial  approach  in  giving  data  feedback. 

As  noted  above,  feedback  provided  by  Likert  scales  in  instruments  keyed 
to  a  specific  format  theory  of  organizational  behavior  may  tend  to  bias  OE 
programs  toward  the  assumption  that  there  is  a  one  right  managerial  style  or 
climate  for  optimum  organizational  functioning.  Contingency  systems  theory 
suggests  this  is  unlikely  to  be  true.  Experience  further  suggests  that  when 
the  theory  being  "pushed"  by  the  survey  is  incompatible  with  military  cli¬ 
ents'  values,  they  are  very  likely  to  reject  the  data  they  receive. 

Absolute  scaling  ("1"  is  "bad,"  “5"  is  "good")  creates  a  pressure  for 
"right"  answers  which  can  bias  survey  results.  When  norm  comparisons  are 
made  (and  sample  size  is  large) ,  supervisors  frequently  get  feedback  that 
shows  (on  the  basis  of  a  .4  variable  difference  from  the  mean)  that  they 
are,  in  the  words  of  one  E8,  "a  double  asterisk,  £  <  .01,  statistically 
significant  incompetent  military  officer."  Feedback  of  this  type  is  both 
misleading  and  not  particularly  conducive  to  constructive  OE  efforts.  The 
norms  used  for  such  comparisons  are  often  meaningless  because  they  are  not 
disaggregated  by  type  of  organization. 

Normative  bias  problems  can  be  avoided  by  providing  actual  and  ideal 
scores  for  variable  dimensions  and  focusing  data  feedback  interpretations 
and  intervention  planning  on  the  client's  personally  felt  discrepancies 
rather  than  comparison  with  an  abstract  norm. 

Consultant  and  client  training.  A  final  consideration  in  designing 
instruments  to  facilitate  effective  feedback  in  organization  intervention 
efforts  is  the  degree  of  consultant  and  client  training  and  expertise  re¬ 
quired  to  use  the  instrument  or  make  sense  of  feedback  received  from  it. 

It  has  been  observed  that  intervention  methods  which  use  complex  instruments 
require  that  a  majority  of  client  time  and  energy  be  spent  in  survey-concept 
workshops  learning  how  to  interpret  instrument  feedback,  rather  than  in 
problem  solving  or  planning  for  constructive  organizational  change,  the 
presumed  purpose  of  the  exercise.  This  "means-ends  inversion"  suggests 
that  work  environment  questionnaires  and  feedback  procedures  should  be  de¬ 
signed  to  be  as  simple  and  self-explanatory  as  possible. 

Evaluation 


Ideally,  work  environment  survey  instruments  should  be  able  to  be  used 
in  formative  and  sumnative  evaluations  of  military  personnel  programs,  and 
to  provide  management  information  system  data  that  policymakers  can  use  to 
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identify  needs,  essese  programs,  and  track  trends  in  organisational  perfor¬ 
mance  over  time.  Reliable  and  valid  instruments  can  be  used  in  test-retest 
designs  to  estimate  the  effects  of  organization  development  programs.  Use 
of  work  environment  questionnaire  data  in  policy  analysis  (for  example,  to 
identify  personnel  policy  changes  which  could  impact  favorably  on  retention, 
Miasler,  1976),  is  possible  if  instrument  items  are  descriptive  in  nature, 
and  if  administrative  and  confidentiality  problems  associated  with  accumu¬ 
lating  work  environment  and  performance  data  on  individual  units  in  a  cen¬ 
tral  data  bank  can  be  solved.  For  example ,  if  personnel  development  train¬ 
ing  opportunities  are  consistently  found  to  predict  junior  NCO  retention 
in  Army  units  with  different  specialties,  this  data  could  inform  organisation- 
wide  training  decisions. 

The  present  study  suggests  that  as  soon  as  work  environment  data  are 
available  an  a  large  sample  of  an  organization's  members,  top  management 
becomes  interested  in  this  information  and  begins  to  request  it  on  a  regu¬ 
lar  basis,  this  may  interfere  with  organization  development  programs  if 
respondents  perceive  that  their  data  are  being  used  to  evaluate  them,  or  if 
top  officials  insist  that  the  work  environment  questionnaire  be  lengthened 
or  modified  to  provide  data  of  interest  to  policymakers  a  potential  obstacle 
which  Should  be  considered  in  the  instrument's  design  phase. 


111.  INTERVIEWS 


To  corroborate  the  work  environment  instrument  and  the  literature  re¬ 
view,  develop  a  pool  of  face-valid  items,  and  identify  Army  unit  effective¬ 
ness  measures,  individual  and  group  interviews  were  conducted  with  a  sample 
of  Any  enlisted  men  at  two  sites*  Fort  Riley,  Kansas,  and  Fort  Carson, 
Colorado. 


Methodology 

Individual  interviews  were  conducted  with  four  junior  (El-4)  enlisted 
personnel  and  four  of  their  NCO  (E5-8)  supervisors  drawn  from  infantry, 
artillery,  armor,  and  support  military  occupational  specialties  (MOS) ,  a 
total  of  32  respondents  at  each  site  using  a  critical  incident  (Flanagan, 
19S4)  at  behavioral  event  (Hertzberg  et  al. ,  1959  >  Ounnette,  1966 »  Smith  £ 
Kendall,  1963 i  and  McClelland,  1976)  technique.  This  method  asks  respon¬ 
dents  to  describe  in  considerable  detail  specific  incidents  in  which  they 
or  their  organization  tie re  particularly  effective  or  had  particularly  high 
morale,  incidents  when  they  or  their  units  were  particularly  low  in  per¬ 
formance  or  morale ,  and  to  state  the  criteria  by  which  they  evaluated  per¬ 
formance,  effectiveness,  and  morale.  A  distinguishing  advantage  of  this 
technique  is  that  it  elicits  descriptive  information  from  which  actual  be¬ 
haviors  can  be  reconstructed,  rather  than  effectively-biased  interpretations 
or  evaluations  of  organizational  functioning,  interview  protocol  can  be 
coded  empirically,  using  thematic  analysis  scoring  methods  developed  by 
Atkinson  and  McClelland  (Atkinson ,  1958) ,  making  interviews  a  quantitative 
method  equal  in  reliability  (r  ■  .8-. 9)  statistics  equaling  those  of  sur¬ 
veys  and  psychometric  tests  (Klenq>,  Hunger,  £  Spencer,  1977).  Each 
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individual  interview  lasted  approximately  an  hour  and  was  transcribed  ver¬ 
batim.  Figure  4  summarizes  the  sample  design. 

Group  interviews  were  conducted  with  the  two  subjects  previously  inter¬ 
viewed  individually  and  four  peers  drawn  from  the  same  four  MOS  groups  at 
each  site,  a  total  of  96  respondents  using  a  group  version  of  the  behavioral 
event  technique.  Respondents  were  asked  to  think  of  the  best  (highest  per¬ 
formance  and  morale)  units  they  had  served  in,  and  the  worst  (lowest  per¬ 
formance  and  morale)  units,  state  the  criteria  by  which  they  rated  a  unit 
good  or  bad,  and  provide  examples  of  leader  behavior,  job  design,  and  group 
and  organizational  working  relationships  and  conditions  from  either  cate¬ 
gory  in  any  order.  Any  statement  or  description  substantiated  by  a  spe¬ 
cific  incident  or  example  was  recorded  on  a  blackboard  in  the  format  illus¬ 
trated  in  Figure  5.  Group  interviews  lasted  an  hour  to  an  hour  and  a  half 
and  were  quite  animated;  the  recorded  respondent  statements  were  transcribed 
verbatim. 


Content  Analysis  of  Interview  Data 

Individual  and  group  transcripts  were  content-analyzed  by  a  researcher 
who  had  not  conducted  any  of  the  interviews  to  identify  both  organization 
process  variables  and  performance  and  satisfaction  outcome  variables.  The 
individual  interviews  were  examined  first.  A  summary  statement  for  each 
discrete  event  described  by  the  interviewee  was  recorded.  These  summary 
statements  were  then  coded  according  to  the  same  content  schema  used  for 
analyzing  the  instrixnents  (see  Table  3) .  While  the  design  is  straightfor¬ 
ward,  single  events  frequently  contained  a  number  of  different  themes  and 
were,  therefore,  recorded  more  than  once.  Similarly,  in  a  few  instances 
almost  the  whole  interview  revolved  around  a  single  theme.  For  example, 
one  enlisted  man  simply  kept  repeating  in  different  forms  his  dissatisfac¬ 
tion  with  the  Army's  job  assignment  and  job  training  procedures.  These 
factors,  plus  the  general  articulateness  and  cooperativeness  of  the  re¬ 
spondents,  explain  the  variation  in  the  number  of  scoreable  items  per  in¬ 
terview.  Unlike  the  analysis  of  the  questionnaires,  scores  of  interviews 
represented  frequencies  and  not  simply  the  presence  or  absence  of  a  par¬ 
ticular  combination  of  a  process  variable  and  organizational  component. 

For  the  purpose  of  data  analysis,  each  of  the  individual  interviews  was 
also  coded  by  installation  (MOS)  and  rank  of  the  respondent. 

The  group  interviews  were  already  in  a  partially  summarized  form  that 
allowed  a  direct  coding  of  the  responses  according  to  the  same  content 
schema.  Each  group  interview  was  coded  by  installation,  (MOS)  and  the 
average  rank  of  the  group. 

Separate  two-way  analyses  of  variance  were  conducted  for  the  scored 
individual  and  group  interview  data  on  respondent  rank  (EM  vs.  NCO) ,  repli¬ 
cated  within  respondent  across  organizational  component  (Job  vs.  Organiza¬ 
tion  vs.  Work  Group  vs.  Supervisor) .  The  analysis  of  the  individual  inter¬ 
view  data  produced  significant  main  effects  for  rank  on  3  of  the  10 
process  variables.  Ziower  level  enlisted  men  (E1-E4)  were  significantly 
more  concerned  with  support  issues  {£  <  .02) ,  while  NCOs  (E5-E9)  were  sig¬ 
nificantly  sore  concerned  with  teamwork  (£  <  .01)  and  responsibility 
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Figure  4.  Behavioral  event  and  group  interview*  sample  design. 
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*N.B. :  Numbers  without  parentheses  are  Behavioral  Event  interviewees; 

numbers  in  parentheses  are  additional  group  interview  participants. 
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(£  <  .002).  Such  findings  are,  of  course,  reasonable  in  light  of  the  dif¬ 

ferent  levels  of  leadership  responsibility  of  these  groups. 

Similar  findings  were  not  made  in  the  analysis  of  the  group  interviews, 
although  the  effects  were  clearly  in  the  same  direction.  In  the  group  in¬ 
terviews,  NCOs  dramatically  increased  the  significance  attached  to  communi¬ 
cation  and  clarity  and  reduced  the  significance  attached  to  standards,  this 
is  most  readily  seen  in  Table  6,  which  gives  the  rankings  of  the  10  process 
variables  according  to  both  the  rank  of  the  respondent  and  the  interview 
setting. 

An  examination  of  the  rank  order  correlations  suggests  that  it  is  not 
simply  the  interview  setting  that  accounts  for  differences  and  nonreplica¬ 
tion,  for  lower  level  EMs  are  highly  consistent  between  the  two  settings 
(£  *  .97) .  Rather,  it  appears  that  it  is  NCOs  in  particular  who  react  to 
the  change  in  interview  setting  ( £  *  .52)  by  adjusting  the  salience  of  the 
various  process  variables.  However,  the  possible  effects  of  small  sample 
sizes  should  not  be  overlooked. 

To  sum  up,  the  rank  of  the  respondent  does  affect  the  importance  at¬ 
tached  to  at  least  four  process  variables  in  the  individual  interviews, 
although  these  effects  are  not  replicated  in  the  group  interviews.  More¬ 
over  NCOs  appear  to  be  more  sensitive  to  the  interview  setting. 

The  analysis  of  variance  also  indicated  significant  main  effects  for 
the  organizational  components  on  8  of  the  10  process  variables  in  the  indi¬ 
vidual  interviews.  The  referent  components  of  superior  and  organization 
were  largely  responsible  as  the  contexts  within  which  the  majority  of  pro¬ 
cess  variables  were  mentioned.  The  organizational  components  f idled  to 
produce  any  differences  in  the  frequency  with  which  Justice  and  Job  Design 
variables  were  mentioned.  These  findings  were  replicated  in  the  group  in¬ 
terviews,  with  the  exception  that  a  significant  main  effect  was  now  present 
for  the  Job  Design  variable  (£  <  .001) .  The  same  emphasis  upon  the  or¬ 
ganization  and  superior  components  found  in  the  analysis  of  climate  ques¬ 
tionnaires  was  also  present  in  the  individual  and  group  interview  data. 

A  ranking  of  the  10  process  variables  by  the  frequency  they  were  men¬ 
tioned,  by  respondent  rank  and  interview  setting  allowed  some  additional 
comparisons,  particularly  with  the  previously  described  analysis  of  the 
climate  questionnaires  {see  Tables  3  and  7) .  First,  it  is  fairly  clear 
that  differences  exist  between  civilian  and  military  instruments .  Not  only 
do  they  produce  different  rankings  of  the  process  variables  but  also  the 
interview  data  are  consistently  more  closely  related  to  the  aggregated 
military  questionnaire  rankings  of  climate  variables  than  the  aggregated 
civilian  questionnaire  rankings.  Second,  the  rank  correlation  coefficients 
suggest  that  military  questionnaires  are  more  attuned  to  the  perceptions  of 
NCOs  than  they  are  to  lower  ranking  enlisted  men.  Both  these  findings  have 
implications  for  the  design  of  future  instruments,  as  did  the  earlier  find¬ 
ing  of  the  differential  importance  attached  to  at  least  4  of  the  10  process 
variables.  Analysis  of  individual  and  group  interviews,  and  incident  and 
description  data  essentially  replicated  the  10  organizational  process  vari¬ 
able  dimensions  found  in  the  instrument  review.  Examples  of  interview 
statements  scored  for  each  variable  are  provided  below. 


Significant  at  .01  (one  tailed  test). 
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1.  Support:  Scored  when  high  or  low  performance  or  morale  results 

from: 

e  supervisor's  concern  for  his/her  subordinates '  personal  needs 

Example :  "X  know  one  good  sergeant.  When  you  came  to  work ,  you  say 
good  morning  to  him,  he'd  say  good  morning  to  you.  With  this  sergeant, 
we  worked  together  like  one  big  happy  family.  When  something  happened  to 
his  personnel,  he'd  try  to  help,  not  just  do  nothing.  Other  NCOs  try  to 
break  a  man  down,  hassle  people.  He  treats  people  like  they're  men.  You 
can  talk  to  him  straight.  If  a  man  needs  a  day  off  to  do  personal  things, 
he'll  give  it  to  him.  He'll  back  you  up.  Like  during  PT,  a  guy  got  a 

cramp  playing  water  polo  and  swam  over  to  the  side  to  rest.  The  E7  there 

said:  'Man,  why  are  you  slacking  off?  You  got  extra  PT  now.'  He  went 
and  told  our  sergeant.  He  said  to  the  E7:  'Don't  you  be  yelling  at  my 
people.  They  don't  work  for  you.*  He  didn't  make  him  do  extra  PT." 

e  provision  or  absence  of  command  services 

Example s  "Everyone  knows  that  Fort  X  is  just  lousy  duty.  There's  no 
transportation,  so  the  guys  got  no  way  to  get  around;  most  young  enlisted 

can't  afford  cars.  There's  nothing  to  do  on  your  off  time.  Recreational 

facilities  are  inadequate.  There  are  only  two  places  to  go:  the  recreation 
center  and  the  sw laming  pool.  People  use  them  but  not  much.  The  reason 
is  that  they  have  too  many  rules,  and  we  get  fed  up..  For  example,  at  the 
pool,  if  the  attendant  decides  to  take  a  break,  you  have  to  get  out  of  the 
water.  At  the  recreation  center,  you  have  to  sign  everything  out,  like 
sign  out  pool  tables,  and  then  they  only  let  you  play  40  minutes  and  then 
kick  you  out.  There's  only  one  little  bank  way  the  hell  over  on  the  other 
side  of  the  base,  and  it's  only  open  during  the  day  when  most  guys  can't 
get  there.  Then  it's  got  only  four  service  windows,  and  on  payday  you  got 
10,000  people  on  this  post  who  got  to  cash  checks,  and  men  have  to  stand 
in  line  for  hours.  In  Germany,  when  you  get  paid,  each  battalion  has  a 
pay  officer  who  pays  you  in  cash  right  there,  or  an  American  Express  person 
who  would  cash  your  check  and  let  you  buy  German  money  right  there  .  .  . 
medical's  really  bad.  You  can't  get  in  there— officers’  wives  can,  but 
ours  can't.  You  don't  see  them  giving  officers'  wives  the  runaround  on 
appointments.  My  wife  lost  two  babies  (miscarriages).  They  didn't  do 
nothing  for  her.  Finally  X  took  her  to  a  civilian  doctor  in  town  and  paid 
for  it  out  of  my  own  pocket.  That  stinks  'cause  that's  part  of  what  X 
signed  up  for:  the  medical  services.  That's  part  of  my  pay." 

•  provision  or  absence  of  hygiene  factors 

Example:  "Bad  leaders  just  care  about  the  mission,  not  for  the  men. 

For  example,  we  were  out  on  half-tracks  on  the  field,  and  a  couple  of  tanks 
broke  down.  The  rest  of  the  company  kept  right  on  going.  The  CO  told  us 
to  wait.  They  just  continued  with  the  mission.  They  forgot  all  about  us. 

Me  didn't  have  any  chow  for  two  days.  That  kind  of  thing  has  happened  other 
times  as  well.  One  day  when  we  were  in  the  field,  it  was  pouring  rain.  He 
(the  CO)  didn't  let  us  erect  a  tent.  It's  standard  operating  procedure  to 
give  the  men  some  shelter.  This  guy  didn't  have  the  troops  first.  He  didn't 
coamninicate  with  them  or  take  care  of  them.  Finally,  he  got  relieved." 
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2.  Personnel  Development t  Scored  when  high  or  low  performance  or 
morale  results  from: 

•  supervisor's  formal  training  or  informal  coachinrj  of  subordinates 
to  improve  their  Job-related  skills  or  help  them  advance  in  their  careers 


Example:  "x  came  from  a  unit  in  Germany  that  was  run  as  an  artillery 
unit  was  supposed  to  run.  It  was  very  effective.  We  were  first  in  the 
AIWEP  inspection  and  on  our  battalion  ARTEP  we  were  judged  most  effective. 
When  I  took  over  in  Germany  my  gun  crew  was  in  last  place.  In  one  field 
trip  one  month,  I  moved  them  all  the  way  to  the  top.  Charlie  battery  was 
the  best  battery  and  my  section  was  the  best  section.  Yet  when  1  took  it 
over,  it  was  known  as  the  'drug  battery*  because  they  had  lots  of  drug 
problems.  I  did  lots  of  counseling.  Z  trained  them  on  the  weapons  they 
were  going  to  use.  The  first  thing  I  always  do  is  find  out  what  type  of 
training  a  guy  needs.  First  Z  explain  the  duties  of  the  cannoneer,  what 
the  specific  job  is.  Then  Z  explain  what  each  man  in  the  crew  does  and 
what  vehicles  are  involved,  then  we  go  over  general  subjects:  weapons 
orientation,  50-caliber  machine  gun,  other  crew-served  weapons.  Then  Z 
take  the  men  out  on  the  gun  itself,  and  go  over  sites,  amnunition  and  safe¬ 
ty.  I  get  them  actually  working  on  the  gun  on  two  or  three  practice  trips, 
going  through  each  step  slowly.  The  fourth  time  out  we  start  working  speed. 
Z  get  people  their  drivers'  licenses  and  drivers'  training.  They  need  this 
because  gun  carriers  don't  drive  anything  like  cars,  and  it  gets  them  eli¬ 
gible  for  promotion  to  Spec.  4.  .  .  .  When  someone  leaves  here,  he  has  a 
very  good  knowledge  of  basic  artillery." 


e  subordinate's  opportunity  to  use  his/her  skills  or  training  or 
feel  competent  and  adequately  trained  to  do  the  tasks  s/he  is  assigned 

Example :  “My  top  sergeant,  he's  a  good  guy.  He'll  sit  around  and  talk 
with  us  about  work.  We  can  tell  him  what  we  did  and  what  went  wrong,  so  he 
can  help  us  straighten  it  out.  When  Z  got  here,  we  were  a  new  unit.  We  got 
classes  every  day  on  our  MOS  so  we'd  know  it  better.  That  was  one  good 
thing.  And  our  officers  were  out  with  us  every  day,  so  they  knew  what  was 
going  on  so  they  could  help  us.  Mow  we  know  our  jobs,  and  do  them  good; 
it's  good  for  a  man  to  know  what  he  can  do.” 

3.  Teamwork}  Scored  when  high  or  low  performance  or  morale  results 

from: 

e  friendly  social  interaction 

Example:  "The  best  top  sergeant  Z  ever  had,  he'd  get  out  and  socialize 
with  us.  ZdJce  after  work,  he'd  come  over  and  have  a  beer  with  us.  He'd 
have  guys  over  to  his  house  for  dinner;  he  did  that  with  everyone  at  least 
once.  He'd  get  you  out  of  the  barracks  for  a  night.  After  an  inspection 
or  when  we'd  had  a  big  push  on,  he'd  organize  a  beer  bust  over  at  the  field 
for  the  whole  section.  He  was  real  strict.  He  pushed  and  pushed  to  get 
the  best.  But  he'd  joke  with  us  even  though  he  pushed.  Zt  really  helps  to 
have  someone  you  know  a  little,  that  treats  you  like  a  human  being.  He 
only  had  him  three  or  four  months,  but  while  we  had  him,  he  got  morale  up 
real  high." 
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•  work  facilitation  ( cooperation ,  mutual  help,  and  teamuork  in  ac¬ 
complishing  group  tasks,  as  opposed  to  conflicts  or  lack  of  cooperation) 

Example ;  "Maintenance  works  when  everyone  pulls  together.  Take  my 
company:  We  got  two  sections,  auto  maintenance  and  artillery  maintenance. 
There  used  to  be  pride  in  the  whole  company,  it  didn't  make  any  difference 
what  section  or  MOS  you  were  in;  you  cooperated,  you  were  connected.  If 
you  needed  parts  or  tools,  you  could  get  them  from  other  sections,  lhat's 
not  true  now.  For  example,  there  is  this  turret  mechanic.  He  had  14  extra 
tools  in  his  tool  box.  The  motor  pool  guy  asked  him  to  borrow  a  tool.  He 
said,  'Why  should  I  give  it  to  you?  X  got  it  and  I'm  going  to  keep  it. 

You  don't  help  me,  and  I  don't  help  you.'  That's  the  way  it  is  now." 

•  group  cohesiveness,  actions  to  create  symbols  of  unit  identity  and 
pride 


Example :  "One  thing  our  new  CO  did,  he  got  us  all  scarfs  with  our  unit 
[insignia]  on  them,  kind  of  to  make  us  feel  special.  When  we  run  PT,  in¬ 
stead  of  just  double-timing  when  we  came  up  behind  another  unit,  our  CO  says, 
'Let's  try  and  pass  them!'  And  he  got  a  little  pride  going..  He  ran  with 
us,  too,  and  when  we'd  come  up  behind  another  unit,  everyone  would  start 
yelling,  'Pass  them,  pass  them!'  He'd  really  like  that.  One  thing  that 
makes  morale  higher  is  if  the  CO  has  pride  in  the  unit." 

4.  Conmmications  Clarity:  Scored  when  high  or  low  performance  or 
morale  results  from  supervisors  or  the  organization  clearly  communicating 
goals,  standards,  and  procedures,  or  from  subordinates  feeling  that  verti¬ 
cal  and  horizontal  communication  channels  are  open  and  that  they  know  about 
situations  concerning  them. 

Example:  "What  brings  morale  down  is  not  really  being  informed,  not 
knowing  what's  coming  off.  It  happens  in  every  unit.  For  example ,  one 
time  I'd  been  in  an  outfit  for  one  month,  in  an  armor  unit  in  Germany.  We 
were  scheduled  to  go  on  adventure  training,  a  raft  trip  down  the  Shine. 

Then  somehow  the  unit  was  assessed  and  picked  for  a  training  exercise  with 
German  nationals.  We  had  to  change  the  adventure  training  to  a  month  or 
so  later.  Morale  really  dropped.  But  when  we  found  out  that  we  had  been 
specifically  picked  for  the  training  with  the  Germans,  morale  went  back  up. 
The  problem  is  we  weren't  told  why  we  were  picked  until  the  exercise  was 
over.  That's  what  hurt  morale.  It  hurt  morale  until  the  troops  found  out. 
Just  getting  told  to  do  something  on  the  spur  of  the  moment  tears  a  person 
down.  Once  it  was  explained  to  us,  when  we  and  the  officers  were  called  in 
and  briefed  and  we  knew  why,  it  was  okay.  Once  we  got  the  word,  we  talked 
to  our  people  and  explained,  and  then  there  was  no  problem.  We  just  loaded 
19  and  moved  out." 

5.  Order  and  Organization :  Scored  when  high  or  low  performance  or 
morale  results  from  effective  or  ineffective  organization  or  logistical 
support  (e.g.,  the  provision  of  needed  materials,  equipment,  and  resources). 

Example:  "Z  was  at  one  place  for  desert  training.  It  was  so  disor¬ 
ganized  we  never  knew  where  we  were  supposed  to  be.  When  you  found  out, 
they'd  tell  you  two  seconds  before  you  were  supposed  to  be  there.  When  you 
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got  than,  the  wrong  people  were  there.  Our  leaders  didn't  know  what  was 
going  on.  Me  were  supposed  to  start  trials  at  6:00  a. a.  but  they  were  not 
ran  until  about  noon,  so  we  were  just  sitting  In  tanks  for  hours  doing 
nothing.  Me  had  big  problems  getting  parts  for  equipment.  We  put  in 
forme,  but  when  we  need  them  we  can't  get  them,  and  we  only  get  them  when 
we  don't  need  them.  For  example,  a  grease  gun  needed  for  lubricating  the 
rood  wheels:  Xt  took  me  seven  months  to  get  a  new  one  after  X  turned  the 
old  one  in.  Another  example,  in  March,  1976,  X  turned  a  part  in,  a  top 
bracket  that  prevents  the  eight  from  swiveling  around.  X  got  it  back  a 
pear  later  and  it's  still  broken.  X  talked  to  my  platoon  sergeant  and  he 
talked  to  the  maintenance  officer  but  nothing  happened.  X  even  went  to 
the  maintenance  battalion  to  talk  to  the  NCO  there:  Initially  X  bugged 
him  once  a  week,  but  finally  X  gave  up.  There's  no  planning  or  coordina¬ 
tion  or  anything;  it's  so  screwed  up  we  can't  do  our  jobs,  and  no  one  seems 
to  care.” 

6.  Standards:  Scored  when  high  or  low  performance  or  morale  results 
from  the  presence  or  absence  of  clear  performance,  personal  conduct,  disci¬ 
pline,  or  equipment  standards. 

Example:  ”X'm  not  happy  with  the  standards  here.  The  division  ser¬ 
geant  major  in  my  last  command  put  out  standards:  'Your  troops  will  look 
like  this,  belt  buckles  shined,  webbing  straight.'  Standards  started 
out  at  the  top  and  worked  their  way  down  until  everyone  knew  them.  Here 
no  one  pushes  standards.  I'm  looked  at  as  a  weirdo  because  X  make  my 
troops  look  sharp.  Here  no  one  on  top  cares,  or  is  pushing  for  setting 
standards  on  how  the  division  should  look.  My  platoon  (me  and  my  lieu¬ 
tenant)  demand  discipline:  no  talking  in  formation,  standing  at  good  at¬ 
tention  position,  just  basic  Army  discipline.  In  other  formations,  guys 
will  be  talking,  scratching  themselves,  looking  around.  Me  and  my  lieu¬ 
tenant  got  here  at  the  same  time.  When  we  first  got  here,  we  looked  over 
the  22-10  manual.  We  looked  at  indicators  of  leadership,  morale,  and 
esprit  de  corps  and  we  saw  all  of  them  were  lacking.  The  unit  was  down 
range  practicing  for  the  ARTEP.  Morale  was  way  down  because  the  battalion 
had  been  down  range  for  throe  weeks  in  winter  just  practicing,  not  for 
real  ARXEP,  just  practice.  We  set  some  standards  for  appearance,  equip¬ 
ment,  practice,  to  give  them  something  to  shoot  for.  We  started  building 
morale  up.  The  unit  didn't  even  look  like  a  regular  Army  unit.  We  did 
it  by  looking  at  a  soldier  and  saying,  'Look  at  your  webbing,  your  uni¬ 
form,  your  haircut.  Xt  should  look  like  this.'  They  started  calling  us 
OXs,  drill  instructors.  But  the  harder  we  were,  the  more  they  respected 
us  and  morale  went  up.  Mow  we're  real  hot.  We  were  rated  the  best  platoon 
in  the  division  on  the  last  ARTEP  inspection." 

7.  Job  Dmmigm  Scored  when  high  or  low  performance  or  morale  results 
from  intrinsic  aspects  of  the  job:  enough  activity  to  keep  people  busy, 
challenging  work,  variety,  identify  with  a  completed  whole,  or  feedback 
from  the  work  itself. 

o  activity 

Example:  "The  worst  unit  X  was  ever  in  was  a  total  waste  of  a  tour. 

Xt  was  a  new  outfit,,  a  cavalry  unit.  They  haul  brand  new  equipment,  but 
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wouldn't  run  it.  The  artillery  just  sat  there.  Their  only  interest  was 
in  infantry  exercises.  All  we  did  was  just  sit  there  for  a  year,  we 
didn't  even  go  to  the  antor  pool  where  our  equipment  was.  X. didn't  ac¬ 
complish  anything.  We  didn't  have  any  ARTEPs,  tests,  feedback  of  any 
kind.  It  was  the  sourest  tour  I've  had  in  21  years  in  the  Army,  because 
X  did  nothing.  And  we  had  brand  new  guns  and  X  could  only  get  to  the 
motor  pool  twice  in  the  whole  year.  We  had  no  goal,  no  mission,  nothing. 

We  never  fired  even  one  round.  X  was  actually  happy  to  go  to  Vietnam: 
at  least  X  was  back  doing  something.” 

e  challenge 

Example:  "Here  at  Fort  X,  there's  no  challenge.  In  Panama,  we  had 
a  real  sharp  unit.  We  had  lots  of  things  to  work  with.  In  training,  there 
was  more  challenge:  you  could  learn.  In  Panama,  training  was  in  the  field. 
We  were  always  in  the  interior,  in  jungle  training.  Scope  training  was 
realistic,  really  like  combat.  We  really  went  places;  there  was  adventure 
in  it.  Here,  you  don't  get  to  go  to  the  field;  you  just  stay  on  the  base 
and  sit  around  in  the  motor  pool  doing  nothing.  Or  if  you  do,  it's  just 
riding  around  in  half-tracks.  People  just  lay  in  the  back  of  the  tracks 
sleeping.  You  don't  feel  any  war  in  it,  there's  no  challenge.  X  dig 
adventure  training;  jungle  training  is  the  most  beautiful  fighting  in 
the  world.” 

e  intrinsic  identity  and  feedback 

Example:  "In  our  ARTEP  results,  we  were  the  best  platoon  in  three 
battalions.  Ours  was  the  only  one  that  fired  rounds.  In  gunnery  we  quali¬ 
fied  four  or  five  tanks,  which  was  the  best  on  the  post.  The  inspection 
team  mentioned  that  to  my  CO.  And  this  was  despite  the  fact  that  three  of 
five  of  the  tank  commanders  were  on  leave  during  the  inspection.  The  men 
did  it.  All  19  were  real  good.  What  X  recommend  to  get  this  good  per¬ 
formance  is  more  time  with  the  tanks,  more  gunnery  practice,  more  good 
tank  work.  Xt's  important  to  keep  the  troops  busy  with  something  real 
that  needs  to  be  done,  not  just  make-work.  X  always  have  something  real 
for  them  to  do.  They  look  forward  to  going  to  the  field.  They  always 
want  to  go,  instead  of  being  in  the  motor  pool  with  the  tanks,  not  moving. 
They  enlisted  to  ride  tanks.  You  need  to  let  them  ride;  otherwise  they 
get  bored. 

"Another  critical  area  is  the  down  range  part  of  the  job.  x  get 
their  hands-on  experience  every  chance  X  can.  People  basically  enjoy  it, 
getting  ready  for  ARTEP,  firing  practice,  just  practice  firing  the  guns. 
Guys  love  it  because  that's  what  they're  getting  paid  for,  and  they're 
good  at  it.  They  make  mistakes,  but  when  that  happens  they  get  told  about 
it.  But  also  when  they  do  it  right,  when  all  four  guns  hit  the  target 
and  the  down  range  spotter  calls  up  and  says,  'Beautiful  sheaf,'  that  goes 
down  the  chain  of  cosmand  to  the  gun  crews  and  they  love  it.  That's  nice. 
They  like  the  recognition.  When  they  screw  up,  well,  the  key  word  is 
'fair.'  They  know  they've  goofed  and  thuy  know  why.  C  battery  has,  in 
the  ten  months  I've  been  there,  consistently  done  better  than  the  other 
battalions." 


8.  Reaponaibili ty  (Autonomy  and  Influence) t  Scored  when  high  or  low 
performance  or  morale  results. from  people's  ability  to  make  job-related 
decisions  without  over-control  from  supervisors,  or  to  feel  they  can  in¬ 
fluence  and  control  their  work  situation. 

— Mf1*’  "The  major  point  is  the  feeling  of  un trustingness  projected 
on  MCOs.  Senior  NCOS  were  doing  everyone  else's  job.  If  everyone  were 
doing  his  own  job,  the  unit  would  run  smoother.  For  example,  the  Amo 
section  has  an  E5  in  charge  and  he's  fairly  competent .  Before  the  last 
XG,  an  E7  who  knows  nothing  about  Ammo  took  it  upon  himself  to  take  over 
the  E5's  section,  preventing  the  E5  from  doing  his  job.  The  E7  is  a  hard 
sun  to  get  along  with.  No  one  likes  him.  The  Ammo  section  worked  12 
hours  a  day  for  two  weeks  before  the  IG,  painting  trucks  to  make  them 
look  nice,  but  they  still  failed  the  inspection.  X  think  it  was  hostility 
casting  out.  The  E5  is  completely  demoralized.  He  was  just  so  ineffective 
when  the  E7  came  over  and  took  his  job  away  from  him,  he  was  just  like  an 
B2  because  he  didn't  have  the  responsibility  to  get  the  job  done.  Now  he 
has  a  very  bad  attitude  and  is  getting  out  of  the  Army. 

"Another  example  is  that  every  time  we  go  to  the  field  we  have  to  do 
basic  periodic  tests,  aligning  the  gun  to  make  sure  it  fires  straight.  The 
E7  has  so  demoralized  the  NCOS  that,  when  I  told  this  one  NCO  to  get  all 
the  stuff  we  needed  for  the  test,  he  got  everything  except  one  thing,  a 
20-foot  piece  of  string.  He  came  to  me  and  said  'I  can't  do  it  because 
I  don't  have  a  piece  of  string.'  Now,  he's  not  that  dumb.  His  initiative 
has  been  completely  zapped.  Hopefully  this  will  change.  The  new  battalion 
CO  has  said  to  the  whole  battalion,  'Section  chiefs  will  have  complete  re¬ 
sponsibility  for  their  units.’  Xf  he  means  that,  it  will  really  help.” 

Examplet  "One  night  we  were  in  the  field.  I  was  driving  the  truck. 

Our  lieutenant  was  with  me.  He  had  lost  the  map.  He  told  me  to  go  straight 
across  the  field.  X  said  to  him,  'Look,  sir,  Z  know  there's  a  big  ditch 
out  there.  X  can't  see  it  in  this  dark  and  it's  real  big.  If  I  go  over, 
all  the  other  trucks  are  going  to  go  over  with  me.  Can  I  go  as  far  as  I 
think  we  can  and  then  turn  and  go  on  the  road?'  He  said,  'You're  in 
charge.'  For  the  next  three  hours,  X  was  leading  the  whole  platoon,  an 
El  telling  guys  who  had  been  in  18  years  what  to  do  and  where  to  go.  X 
was  right.  The  ditch  was  right  where  X  thought  it  was.  That  really  put 
my  morale  up  because  X  had  real  responsibility." 

9.  Feedback  and  Reward* :  Scored  when  high  or  low  performance  or 
morale  results  from  supervisors,  co-workers,  or  the  organization  giving 
individuals  feedback  on  how  well  they  are  doing  their  jobs,  or  rewarding 
good  task  performance. 

"i  came  from  Hawaii,  from  the  25th  division.  There  we  had 
the  best  division  in  the  Army.  We  really  had  it  all  together.  When  X 
first  got  here,  X  started  coegwring  Fort  X  with  the  25th.  The  first  thing 
X  saw  was  that  there  was  no  ascendency  program  for  soldiers.  The  25th  had 
a  "bsst  by  test"  program  to  recognize  the  good  soldier i  gunner,  rifle 
squad,  every  specialty.  Bach  level,  platoon,  then  company  level,  then  at 
battalion  level,  identified  its  best  soldiers  in  each  specialty.  At  each 
level,  there  was  a  competition  for  the  best  policeman,  rifleman,  etc. 

Guys  Who  won  got  public  recognition «  a  certificate,  a  three-day  pass. 
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that  had  norale  sky  high.  It  really  means  something  to  a  soldier  to  have 
a  coamendation  from  the  division  commander ,  a  two-star  general .  They 
worked  hard  over  there,  but  they  walked  around  real  proud.  There  *  s  noth¬ 
ing  like  that  at  Fort  X,  and  there  should  be.  The  most  important  thing 
is  to  let  guys  know  how  they  are  doing,  and  give  the  deserving  ones  a  lit¬ 
tle  pat  on  the  back.  Most  people  want  to  do  a  good  job,  if  you  show  them 
how.  I  have  a  little  book  and  I  keep  track  of  who's  doing  a  good  job. 

For  the  guys  who  are  doing  a  good  job,  if  something  comes  up  and  is  up  to 
standards.  I'll  cut  them  loose.  If  someone's  not  performing  as  he  should, 

I  deal  with  him  by  personal  counseling.  I  have  this  one  E6  who's  slow  on 
the  uptake,  but  he's  improving.  I  sat  down  with  him  and  let  him  know  where 
he's  weak  and  how  he  can  improve .  it's  hard.  I  hate  to  do  it.  I  just 
start.  I  have  a  5x8  card  and  I  just  start  by  saying  how  I  feel  and  I  go 
down  my  list.  I  tell  him  the  things  he's  not  doing  up  to  par  and  I  let 
him  ask  questions.  Then  I  give  him  a  pep  talk  about  what  I  expect,  and 
how  he  can  square  away,  and  what  will  happen  if  he  doesn't  square  away. 

I  deal  with  people  as  individuals.  I  try  to  let  him  know  right  away  where 
he  stands.  Some  I  pat  on  the  back,  some  I  kick  in  the  ass,  depending  on 
the  person." 

10.  Justice.*  Scored  when  high  or  low  morale  results  from  the  extent 
to  which  rewards,  discipline,  and  work  assignments  are  handled  on  a  fair 
and  equal  basis,  without  discrimination  or  favoritism  by  race,  sex,  or 
other  individual  bases. 

Example;  "With  discipline,  there  are  problems  with  favoritism.  One 
guy.  I'll  call  him  John  Doe,  had  a  little  personality  dispute  with  the  co 
and  missed  a  formation.  He  got  an  Article  15  and  got  14-14  (that  is,  14 
days  restriction  and  14  days  extra  duty)  and  a  fine.  Now  another  guy  went 
AMGL  and  left  a  weapon,  a  loaded  M16  rifle,  unattended.  That's  a  major 
violation .  I  found  the  M16  and  turned  it  in  and  I  wrote  up  a  charge  sheet 
on  him.  Be  went  in  for  an  Article  15  hearing  and  he  got  seven  days  extra 
duty,  seven  days  restriction  and  a  small  fine,  and  all  of  them  were  sus¬ 
pended.  He  was  the  teacher's  pet.  He  just  got  a  slap  on  the  wrist.  The 
CO  screwed  up.  The  troops  didn't  come  right  out  and  say  it,  but  you  could 
feel  it.  They  felt,  'One  guy  goes  AWOL  and  leaves  an  M16  out,  a  big  thing, 
and  gets  all  of  his  punishment  suspended.  The  other  guy  just  misses  for¬ 
mation  and  gets  busted.'  That's  favoritism  and  it  hurts  morale." 


IV.  ORGANIZATIONAL  EFFECTIVENESS  CRITERIA 


The  third  subtask  of  Phase  I  of  this  study  was  to  identify  organiza¬ 
tional  effectiveness  criteria,  "hard"  result  outcomes ,  that  could  be  used 
to  measure  the  effects  of  work  environment  and  organizational  intervention 
efforts.  Four  methods  were  used  to  identify  these  criteria:  (1)  content 
analysis  of  the  behavioral  event  interview  protocols,  (2)  direct  question¬ 
ing  of  respondents  in  the  group  interviews  about  effectiveness  measures 
used  by  their  wits  or  routinely  reported  in  installation  management  infor¬ 
mation  systems,  (3)  similar  questioning  of  QESOs  at  the  two  interview  sites 
about  criteria  they  used  or  would  like  to  see  made  available  to  assess 
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organizational  affectiveness,  and  (4)  tabulation  of  measures  reported  in 
the  criterion  validity  studies  summarized  in  the  instrument  review. 

Consistent  with  the  instrument  review  taxonomy,  effectiveness  measures 
were  categorized  as  either  performance  or  satisfaction  outcomes.  The  cri¬ 
teria  identified,  with  brief  ccunents  on  their  availability  and  reliability 
as  perceived  by  respondents,  are  summarized  below.  These  criteria  should 
be  considered  as  a  "shopping  list"  of  alternatives  rather  than  a  recom¬ 
mendation  of  specific  measures.  As  a  result  of  the  Army's  highly  decen¬ 
tralized  management  system,  the  availability  of  measures  varies  widely 
from  Installation  to  installation.  In  most  cases  OESOs  will  have  to  de¬ 
termine  the  availability  and  reliability  of  organizational  effectiveness 
measures  on  a  site-by-site  basis. 


Performance  Measures  as  Perceived  by  Respondents 

Five  types  of  performance  effectiveness  criteria  were  found:  inspec¬ 
tion  scores,  mission  accomplishment  results,  efficiency  measures,  personnel 
development  indices,  and  awards. 


Inspection  Scores 

1.  ARTEP  scoras:  the  most  commonly  cited  combat  preparedness  rating. 
|  Available  at  the  unit  (e.g.,  platoon  battery)  and  battalion  levels.  Relia- 
■  bility  considered  good. 

2.  IG  inspection  scores:  Scored  2  •  outstanding,  1  »  satisfactory, 

0  *  unsatisfactory.  Second  most  commonly  cited  rating.  Available  for  all 
units  at  most  levels  on  a  variety  of  criteria:  equipment  maintenance, 
materials,  inventory,  space  cleanliness,  personnel,  etc.  Reliability  con¬ 
sidered  only  fair  because  IG  inspectors  are  reportedly  "reluctant  to  give 
Onsets  because  they  can  wreck  a  guy's  career,”  so  they  give  mainly  "Sats" 
with  a  verbal  warning  about  what  to  fix;  hence  there  is  a  great  variation 
among  units  with  identical  "Sat"  scores. 

3.  TPI  (Technical  Proficiency  Inspection)  scores :  nuclear  weapons 
use  and  security.  Available  at  battalion  and  installation  levels  for  ar¬ 
tillery  units  qualified  for  nuclear  weapons.  Reliability  considered  very 
good,  although  scores  perceived  largely  depending  on  "security  paperwork" 
(e.g.,  one  command  reportedly  failed  because  an  office  clerk  had  not  had 
a  drug  urinalysis  recorded  on  a  personnel  form) . 

4.  CONST  scores i  Department  of  the  Army  quarterly  inspection  of  all 
equipment  except  TASOs  (equipment  issued  to  individuals) .  Available  in 
some  units  (other  respondents  had  never  heard  of  this  inspection) .  No 
data  on  reliability. 

5.  TA50  scores t  possession  of  equipment  issued  to  individuals. 
Availability  depends  on  units:  Some  keep  strict  records,  hold  formal 
inspections;  others  appear  more  casual.  Reliability  questionable  if 
data  collection  is  informal. 


84 


6.  SQT  scores  (Special  Qualifications  Tests):  practical,  hands-on 
tests  on  individual  competence  in  use  of  actual  equipment.  Available  for 
seem  MOSs.  Reliability  considered  very  good  if  test  is  practical  and  on 
specific  equipment  (e.g.,  the  time  taken  to  arm  an  actual  mifte)  but  unre¬ 
liable  if  a  paper  and  pencil  test  ("a  lot  of  guys  who  get  high  scores  on 
those  tests— and  get  promoted — don ' t  know  which  end  is  up  on  an  actual 
piece  of  equipment  when  they  get  in  the  field") . 

7.  Field  Day  scores:  a  variety  of  measures  available  at  unit,  com¬ 
pany,  and  battalion  levels  where  field  days  are  held — e.g.,  tank  placement, 
firing  scores,  time  to  run  an  obstacle  course  for  armor  units.  Availa¬ 
bility  seems  at  the  option  of  the  units:  Some  have  field  days  and  measure 
results;  others  do  not.  No  data  on  reliability;  probably  varies  widely. 


Mission  Accomplishment  Results 

1.  Number  of  hours  flown  in  air  units:  available  and  reliable. 

2.  Accident  rates:  equipment  or  personnel  casualties  per  given 
period  of  time  (1,000  hours  of  flying  time,  one  month  or  one  year) .  Avail¬ 
able  and  reliable  where  records  are  kept. 

3.  Mission  objectives:  e.g.,  the  number  of  objectives  in  a  command's 
"management  by  objectives"  plan  accomplished  to  standards  within  the  time 
period  projected.  Mission  objectives  are  highly  variable  by  organization; 
availability  and  reliability  are  questionable  in  most  units. 


Efficiency  Measures 

1.  Operational  Readiness  Status  Reports  ("OR  rates"):  measure  of  the 
status  of  each  piece  of  equipment,  whether  it  is  functional  and  combat 
ready  or  nonqperational .  Available  in  some  armor  units,  motor  pools,  and 
the  like.  Reliability  considered  poor  because  "much  gundecking  goes  on." 

2.  Deadline  reports:  measures  of  the  time  required  to  get  a  given 
piece  of  equipment  repaired.  Available  in  some  maintenance  groups.  Relia¬ 
bility  varies  greatly  with  the  strictness  of  recordkeeping  among  units. 

3.  Equipment  casualty  reports:  special  report  chits  which  must  be 
submitted  when  certain  equipment  breaks  down. 

4.  Maintenance  requests  (Form  2404) :  measure  of  time  elapsed  between 
the  time  a  request  is  made  ("time  in"  noted  on  the  request  chit)  and  the 
maintenance  task  is  completed  and/or  the  repaired  piece  of  equipment  is 
returned  to  its  unit  owner  ("time  out"  on  request  chit) .  Available  in  some 
maintenance  groups .  Reliability  varies  v.’ith  the  unit's  recordkeeping 
procedures . 

5.  Parts  requisitions:  (1)  measure  of  time  elapsed  from  the  time  a 
part  is  ordered  or  requested  and  the  time  received,  and  (2)  "completion 
rate":  the  percentage  of  correct  parts  received  to  total  parts  ordered; 
hence  a  measure  of  errors  (delivery  of  the  wrong  part) ,  stock  outs,  broken 
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parts  racaivad,  ate.  Available  in  supply  units;  reliability  varies  with 
unit  recordkeeping. 

6.  Equipment  lost  reports:  available  by  category  in  most  units.  Re¬ 
liability  poor — much  "slippage"  and  "gundecking"  perceived  by  respondents. 

7.  Service  requests:  measures  of  (1)  the  time  elapsed  between  the 
time  a  service  is  requested  and  the  time  it  is  received  (an  installation 

or  MOS  reassignment,  housing  placement,  medical  appointment,  a  check  cashed, 
or  a  report  typed) ,  and  (2)  "completion  rate" :  the  percentage  of  service 
requests  completed  to  the  requestor's  satisfaction  to  the  total  number  of 
service  requests.  Rarely  available;  no  data  on  reliability. 

Note:  Service  request  measures  are  particularly  relevant  for  hard-to¬ 
me  asure  installation  support  functions  performed  largely  by  civilian  civil 
service  or  contractor  personnel.  For  example,  the  organizational  effective¬ 
ness  of  electricians,  plumbers,  secretaries,  personnel  and  housing  office 
clerks,  and  the  like  can  all  be  measured  with  appropriate  service  request 
data.  These  data  can  readily  be  translated  in  cost  and  benefit  dollar 
figures  by  multiplying  time  statistics  by  hourly  wages  or  wage  equivalents 
(direct  labor  expenses  plus  fringe,  overhead,  general  and  administrative, 
and  in  the  case  of  civilian  contractors,  fee,  amounts).  Service  request 
data  do  not  seem  to  be  available  at  most  Army  installations,  but  it  should 
be  relatively  easy  to  implement  management  information  system  to  collect 
data  of  this  kind. 

8.  Cost  budget^:  whether  officers  with  budgets  exhaust  their  funds 
before  the  end  of  the  budget  period,  overrun  allotted  budgets,  or  have  un¬ 
expended  funds.  Several  respondents  noted  that  they  received  a  specific 
dollar  or  material  budget  (e.g.,  "I've  got  only  125  rounds  of  ammo  to  last 
we  a  year") ,  and  that  "affective"  supervisors  and  organizations  planned 
carefully  to  stay  within  these  constraints  and  balance  expenditures  over 
the  allotment  period,  while  "ineffective"  officers  "used  everything  up  in 
the  first  four  months  and  couldn't  do  any  training  after  that."  Availa¬ 
bility  and  reliability  of  unit  cost  and  budget  information  are  likely  to 
be  major  problems  with  this  measure. 


Personnel  Peveloanent  Measures 

1.  Promotions:  the  percentage  of  persons  promoted  (or  eligible  for 
promotion  by  test  and  point  scores)  at  important  rank  levels  (e.g.,  E3  to 
E4  to  the  total  number  of  persons  at  the  lower  rank) .  This  measure  is 
frequently  used  as  an  equal  opportunity  indicator  by  comparing  the  promo¬ 
tion  percentages  of  majority  vs.  minority,  or  male  vs.  female  personnel, 
and  can  be  used  as  an  indirect  measure  of  supervisor  and  unit  training  ef¬ 
fectiveness.  Data  available  and  reliable. 

2.  Education:  percentages  of  persons  taking  or  completing  required 
or  optional  courses:  correspondence,  advanced  MOS  training,  GED  high  school 
equivalency,  college  courses,  etc.  Available  for  some  units.  Reliability 
good  where  data  are  kept. 
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3.  Physical  training :  mean  unit  scores  or  percentages  of  people 
passing  required  PT  tests.  Available  in  many  units.  Reliability  good 
(some  "gundecking"  and  "excused  people"  manipulation  reported  by  re¬ 
spondents:  "guys  who  aren't  likely  to  pass  get  medical  profiled  the  day 
of  the  "PT  test"). 


Awards 

1.  Unit  citations:  available  where  base  COs  give  them.  Reliability 
depends  on  basis  of  award:  good  if  given  for  inspection  results,  poor  if 
for  "special  performance— because  there  aren't  any  set  standards";  and 
these  awards  are  considered  capricious. 

2.  Individual  citations  and  merit  ratings:  available  and  reliable 
•specially  where  formal  "best  by  test"  competitions  are  conducted  to  iden¬ 
tify  and  reward  performance  in  MOS  categories. 


Satisfaction  Measures 

Included  as  satisfaction  measures  were  retention,  discipline,  inter¬ 
group  health,  family,  and  community  relations  outcomes,  on  the  assumption 
that  individual  feelings  of  well-being  or  discontent  are  casually  related 
to  these  outcomes. 


Retention  Rates 

1.  Retention  rates:  the  percentage  of  first  term  and  career  reen- 
listments,  or  junior  officer  resignations.  Reenlistment  intentions  are 
frequently  used  as  a  surrogate  variable  because  intentions  (measured  by 
survey  responses  to  the  question  "at  this  time  I  intend  to  reenlist/not 
reenlist")  have  been  found  to  correlate  highly  with  actual  reenlistment 
actions  (ir  ■  .80-. 95,  Gould,  1976;  Wissler,  1976) .  Available  and  reliable. 


Disciplinary  Actions 

1.  Article  15s:  number  per  100  enlisted  men  per  a  given  time  period 
(month  or  year) .  Article  15  actions  can  be  broken  down  by  reason  given  to 
provide  specific  measures:  unauthorized  absences,  drug  use,  insubordina¬ 
tion,  fights,  racial  incidents,  and  the  like.  Available  in  most  units. 
Considered  somewhat  unreliable  and  questionably  interpretable  in  terms  of 
organization  effectiveness  by  most  respondents  because:  (1)  standards  vary 
("one  guy's  tough  and  hands  out  Article  15s  for  every  little  thing,  another 
hardly  ever  writes  anyone  up") ,  and  (2)  many  military  officers  consider 
same  Article  15s  to  be  a  positive  indicator  of  organizational  effectiveness 
("a  few  busts  show  that  a  unit's  got  some  discipline  and  standards  are  be¬ 
ing  maintained"). 

2.  Court  martial  actions:  number  per  100  enlisted  men  per  time  period. 
Available  and  reliable  but  relatively  rare,  hence  perhaps  not  a  good  measure 
of  organizational  effectiveness. 


3.  AMQL  rates ;  the  most  commonly  cited  measure  of  satisfaction. 
Available  and  reliable.  OESOs  noted  that  it  would  be  valuable  to  break 
AMQL  statistics  down  by  cause,  for  exasqile,  family  crises  versus  dis¬ 
satisfaction  with  one's  unit  or  the  Army,  but  indicated  that  this  data 
would  be  hard  to  get  on  a  reliable  basis. 

4.  Drug  charges:  formal  arrests  or  "busts"  per  100  enlisted  men. 
Available  but  considered  somewhat  unreliable  because  some  supervisors  are 
much  harder  on  drug  use  than  others. 

5.  Assaults;  available  but  perhaps  unreliable:  Some  supervisors 
consider  an  occasional  fight  "good  for  the  boys"  and  do  not  report  assaults 
to  the  UCMJ  system.  Serious  assaults  probably  reliably  reported. 

6.  Thefts:  thefts,  measured  by  the  incidence  of  theft  reports,  men¬ 
tioned  to  be  an  indicator  of  satisfaction,  morale,  and  discipline  ("you 
don't  get  as  many  on  well  run  posts  where  morale  is  high").  Available  and 
as  reliable  as  any  "crime  statistics"  (probably  underreported) . 

Mote:  Numerous  other  "crime  statistics"  could  be  used  as  organiza¬ 
tional  effectiveness  measures  where  base  or  community  records  are  kept; 
the  discipline  and  infraction  criteria  mentioned  above  were  the  most  fre¬ 
quently  cited  by  study  respondents .  Seme  infractions — driving  while  in¬ 
toxicated  (DWI)  charges,  child  abuse,  and  inter-group  assaults— are  dis¬ 
cussed  as  separate  criteria  below. 


Health  Statistics 

1.  Accident  rates;  number  per  1,000  hours  worked  or  per  1,000  employ¬ 
ees.  Same  installations  have  excellent  accident  report  record  systems  and 
have  made  accident  rates  a  major  organizational  effectiveness  priority. 
Availability  and  reliability  vary  by  site. 

2.  Sick  calls:  number  of  medical  appointments  requested,  lab  tests 
performed  or  pharmacy  units  dispensed  divided  by  the  total  number  of  per¬ 
sons  in  a  unit  or  at  an  installation.  Medical  data  can  be  broken  down  into 
specific  measures  by  admitting  complaint,  for  example ,  accidents,  venereal 
disease,  urinalysis  positive  for  drug  use,  alcoholism,  stress-related  ill¬ 
nesses  such  as  ulcers  and  hypertension,  and  psychiatric  conqplaints:  anxiety, 
depression,  disturbed  behaviors.  Availability  may  be  constrained  by  pri¬ 
vacy  act  considerations.  Reliable.  Malingering  as  well  as  real  illness 

can  be  considered  an  organizational  satisfaction  measure. 

3.  Suicides:  data  availability  and  reliability  questionable  (sui¬ 
cides  are  reportedly  "hushed  up") . 

4.  Alcohol  abuse:  alcohol  referrals,  including  driving  while  intoxi¬ 
cated  (DWI)  charges,  leading  to  mandatory  treatment  as  measured  by  percent¬ 
ages  of  people  referred  to  total  numbers  in  a  unit  or  at  an  installation 
program.  Available  where  alcohol  programs  exist  and  reliable  (underre¬ 
ported)  ,  but  there  are  questions  of  interpretation:  Do  more  effective 
organizations  identify  and  try  to  help  more  alcohol  abusers,  or  do  poor 
organisations  "drive  people  to  drink?" 


5.  Drug  abuse  referrals :  percentage  of  drug-positive  urinalyses, 
self-ref errals ,  or  mandatory  referrals  as  a  percentage  of  persons  in  a 
unit  or  at  an  installation.  Available  at  installations  with  drug  pro¬ 
grams  and  reliable,  although  with  questions  of  interpretation. 

6.  Psychiatric  complaints!  incidence  of  persons  seeking  counseling 
for  emotional  problems.  Available  but  may  be  unreliable  (one  respondent 
reported  his  CO  had  ordered  that  "no  one  be  sent  to  the  shrink,  it  makes 
the  command  look  bad"). 

Note:  Other  illness  categories  can  be  used  as  organizational  ef¬ 
fective  criteria  if  data  are  available;  those  cited  above  were  mentioned 
most  frequently  by  study  respondents. 


Interqroup  Relations 

1.  Racial  incidents:  incidence  on  or  off  the  base.  Availability 
and  reliability  questionable  because  incidents  are  often  not  reported. 
(One  respondent  gave  the  following  example:  "The  first  CO  who  reported 
a  racial  incident  got  a  call  from  the  Commanding  General  personally— the 
CG  told  him  he  didn't  want  to  hear  of  any  more  race  incidents,  and  sure 
enough,  no  one's  ever  reported  another  one.") 

2.  Equal  opportunity  indicators:  numbers  of  equal  opportunity  com¬ 
plaints  filed  and  percentages  of  minority  groups  and  women  promoted  at 
various  rank  levels.  Increasingly  available  and  reliable  where  formal 
complaints  have  been  filed. 

3.  "Unobtrusive  measures" :  e.g.,  the  number  of  minority  personnel 

in  "white"  bars  and  vice  versa  on  randomly  observed  nights.  Data  of  this 
kind  generally  unavailable,  difficult  to  collect,  and  unreliable. 


Family  Measures 

A  number  of  respondents  mentioned  military  persons'  management  of 
their  personal  lives  and  families  as  indirect  organizational  effectiveness 
criteria.  Measures  included: 

1.  Divorce  rates:  may  be  available  from  personnel  records;  possible 
privacy  concerns. 

2.  Requests  for  family  counseling:  available  and  reliable  where  this 
service  is  provided  on  post. 

3.  Credit  and  indebtedness  indices:  the  incidence  of  persons  seeking 
credit  or  personal  financial  management  counseling,  having  wages  withheld 
to  pay  off  debts,  having  automobiles,  furniture,  or  other  possessions  re¬ 
possessed,  "bouncing"  checks ,  or  filing  for  personal  bankruptcy.  Respon¬ 
dents  noted  that  supervisors  often  issued  Article  15s  to  subordinates  who 
mismanaged  their  personal  finances.  ("You  bounce  a  check  at  the  commissary, 
you  get  restricted.")  Availability  and  reliability  questionable;  possible 
privacy  act  problems. 
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4.  Child  abuse :  incidence  among  members  of  units.  Frequently  cited 
by  respondents  but  probably  unavailable  and  unreliable. 


Community  Relations 

1.  Civilian  arrests;  incidence  per  100  enlisted  personnel,  broken 
down  by  type  of  infraction.  Available  and  reliable  where  military  and 
civilian  police  share  information. 

2.  Community  protests;  incidence  of  complaints  about  military  per¬ 
sonnel  in  civilian  areas.  Probably  not  systematically  available  and 
unreliable. 


APPENDIX  A 


PRINCIPLES  OF  SURVEY  INSTRUMENT  CONSTRUCTION. 


Instrument  Construction 


The  literature  on  survey  instrument  construction  (Hyman,  1955;  Nun- 
nally,  1967;  Babbie,  1973)  provides  a  number  of  criteria  for  assessing 
instrument  construction.  These  include:  scaling,  item  content,  compre¬ 
hensibility,  response  bias,  and  item  and  instrument  rejection. 

Scaling 

Several  different  item  formats  may  be  used  to  elicit  information  from 
survey  respondents.  Four  considerations  are  scale  anchors,  scale  format, 
scale  range  (number  of  scale  intervals) ,  and  scale  neutral  points. 

Scale  Anchors ;  Scale  intervals  can  be  "anchored"  with  horizontal 
lines,  numbers,  statements  such  as  "agree"  or  "disagree,"  interval  defini¬ 
tions  with  specific  data  or  behavioral  criteria,  happy  or  sad  cartoon  faces 
(Smith,  Kendall,  t  Hulin,  1969;  Kunin,  1955)  or  left  unanchored. 

Examples : 


(a)  Specific  Data: 

"The  amount  of  time  I  spend  working  alone  is:" 

I - f - 1 - 1 - - 1 - 1 

0-20%  20-40%  40-60%  60-80%  80-100% 

(b)  Behavioral  Description : 

Which  of  the  following  phrases  best  describes  your  supervisors'  goal¬ 
setting  behavior? 


1 - 

Never 

mentions 

goals 


- 1 - 

Has  vague 
goals, 
nothing 
specific 


- 1 - 

Talks  about 
specific 
goals,  but 
does  not  have 
specific  mea¬ 
sures  of  goal 
accomplishment 


- 1 - 

Sets  speci¬ 
fic,  measur¬ 
able  goals, 
but  does  not 
have  action 
step  mile¬ 
stones  with 
due  dates 


- 1 

Sets  speci¬ 
fic,  measur¬ 
able  goals 
and  action 
step  mile¬ 
stones  with 
due  dates 


The  general  rule  is  that  more  specific  interval  anchor  descriptions 
produce  more  accurate,  less  ambiguous  data  where  a  respondent  has  been 
asked  to  make  a  judgment  about  something  he  can  objectively  observe.  Con¬ 
versely,  unanchored  scales  may  produce  less  biased  data  about  purely  sub¬ 
jective  perceptions  because  the  respondent  is  not  forced  into  choosing  a 
category  and  hence  can  report  finer  nuances  of  feeling. 


traaple: 

Place  a  mark  on  tha  line  below  at  the  point  which  best  indicates  how 
auch  you  like  your  job. 

Very  little  111  -  -  1  — -  — . -  — '"Very  much 

An  additional  consideration  with  anchors  is  to  make  sure  that  inter¬ 
vals  are  equally  weighted.  For  example,  Taylor  and  Bowers  found  that  the 
following  anchors  produced  different  responses: 


Older  Form 

Machine-Scored  Form 

1. 

not  at  all 

1. 

to 

a  very  little  extent 

2. 

to  a  very  little  extent 

2. 

to 

a  little  extent 

3. 

to  seam  extent 

3. 

to 

some  extent 

4. 

to  a  considerable  extent 

4. 

to 

a  great  extent 

5. 

to  a  very  great  extent 

5. 

to 

a  very  great  extent 

In  this  case,  "not  at  all"  in  the  older  form  was  more  negative  than  "to  a 
very  little  extent"  in  the  machine-scored  form,  and  the  lower  end  of  the 
older  form's  scale  was  out  of  balance  with  its  higher  end. 

Most  instruments  reviewed  used  Likert  scales  with  numerical  anchors; 
very  few  provided  specific  descriptive  or  data  anchors  for  item  intervals. 

Scale  Format:  At  least  five  different  item  formats  were  identified  j 

in  reviewing  work  environment  and  organisational  climate  instruments. 

Likert t  By  far  the  most  cannon  item  format  was  the  "Likert  scale," 
usually  a  line  with  five  or  seven  interval  points  below  or  to  the  right 
of  an  item. 

Rramplemt 

(a)  "My  supervisor  sets  specific  goals  for  this  work  group." 

| - 1 - 1 - J - I 

TO  a  very  To  a  To  some  To  a  To  a  very 

little  little  extent  great  great 

extent  extent  extent  extent 

(b)  "Work  activities  are  sensibly  organized  in  this  company." 

| - 1 - 1 - 1 - 1 

Strongly  Mildly  So-so  Mildly  Strongly 

disagree  disagree  agree  agree 

Likert  scales  have  the  advantage  of  being  simple  and  easily  interpretable 
by  respondents  and  of  eliciting  a  range  of  judgments,  but  may  result  in 
biased  data  (see  discussion  under  "response  set"  below) . 

Semantic  Differential/  Scales  can  be  constructed  with  two  diametri¬ 
cally  opposed  item  descriptions  on  each  side  of  a  scale  line.  Respondents 
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are  ukcd  to  mark  the  point  on  the  line  that  best  describes  their  perception 
of  the  variable. 


Mxamplt: 


(a)  Mark  the  point  on  the  line  below  which  best  describes  your  supervisor ' s 
handling  of  disciplinary  actions s 

Fair  - 1 - 1 - 1 - 1 - 1 - Unfair 


(b)  People  do  not 
know  what  is 

expected  of  - 1 - {■ 

then  in  this 
organization 


People  know 
what  is  ex- 

- pected  of 

them  in  this 
organization 


Arguments  for  use  of  semantic  differential  scaling  include  evidence 
that  people  tend  to  think  in  terms  of  the  bipolar  contrasts  of  a  relatively 
small  number  of  characteristics  of  anything  they  observe  or  evaluate  (Os¬ 
good,  1962;  Osgood  et  al.,  1957)  Newell  6  Simon,  1972)  and  that  semantic 
differential  formats  present  respondents  with  an  unambiguous,  visually  and 
conceptually  balanced  choice. 


Torcmd  Choice;  Scales  can  be  constructed  which  force  a  respondent  to 
choose  one  of  two  alternatives,  as  in  the  familiar  "True-False"  format. 


Examplaai 


(a)  "People  who  do  good  work  get  rewarded  in  this  organization." 

True  _ 

False _ 

(b)  "My  supervisor's  handling  of  disciplinary  actions  is»" 

Fair _ 

Unfair _ 


Forced  choice  (dichotomous)  items  have  the  advantage  of  eliciting  unambigu¬ 
ous  data,  but  by  restricting  the  respondents'  alternatives,  they  lose  much 
of  the  power  attainable  with  scaled  items  and  are  less  reliable  than  scales 
with  more  interval  choices.  Item  rejection  may  also  be  a  problem.  "The 
unacceptability  to  the  rater  of  the  forced-choice  format  [is)  the  most 
crucial  deciding  factor.  The  experience  of  the  Army  with  this  system  led 
to  its  abandonment  in  1950,  since  raters  .  .  .  found  it  so  unacceptable 
to  rate  without  knowledge  of  the  final  outcome  that  they  concentrated  on 
finding  ways  to  beat  the  system."  Smith  &  Kendall  (1963). 

Jt naked  .Responses;  items  can  be  constructed  which  ask  respondents  to 
number  response  alternatives  on  the  basis  of  their  agreement  or  disagreement 
with  the  item,  force  rank  alternatives,  or  distribute  a  number  of  points 
between  several  response  alternatives. 
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XxaaplmMt 

(a)  Viaband  responses: 

Using  the  numbers  1  to  5  (1  -  as  little  as  possible  and "5  »  as  much  as 
possible) ,  indicate  how  such  of  the  following  characteristics  you 
would  like  on  your  job  by  putting  the  appropriate  number  in  the  box 
opposite  the  item. 


a. 

the  chance  to  do  interesting  work 

m 

b. 

security 

© 

c. 

a  high  salary 

© 

(b)  Forced  ranking: 

Using  the  number  1  to  indicate  your  first  choice,  2  to  indicate  your 
second  choice,  and  3  to  indicate  your  third  choice,  rank  the  following 
things  you  would  like  from  your  supervisor  by  putting  the  appropriate 
number  in  the  box  opposite  the  item. 

a.  treat  me  in  an  open,  friendly  way  m 

b.  let  me  know  exactly  what  he/she  expects 

of  me  JT] 

c.  give  me  more  responsibility  a 

(c)  Distribution  of  numbers: 

Distribute  10  points  asong  the  following  sentences  to  indicate  how 
often  you  do  each  of  the  following  on  your  job.  Always  use  all  10 
points.  Hever  use  more  than  10  points  or  fewer  than  10  points.  You 

may  use  xeroes. 

a.  work  with  others  inside  the  organization  a 

b.  work  by  aiyself  Q] 

e.  work  with  others  outside  the  organization  fol 

Hanked  responses  tend  to  be  less  easily  interpreted  than  graphic  scales, 
which  provide  a  visual  example  of  the  rating  continuum  and  range  of  response 
alternatives.  As  a  result,  items  of  this  type  produce  more  errors  due  to 
respondent  confusion  [subjects  tend  to  forget  the  meaning  of  the  numbers 
and  confuse  the  ends  of  the  scales  (e.g.,  "Did  *1'  mean  I  agree  or  X  dis¬ 
agree?”)],  mistakes  in  arithmetic  (users  of  distributed  number  scales  are 
frequently  surprised  at  the  number  of  respondents  who  cannot  add  up  to  10) , 
and  clerical  errors  in  transcribing  numbers  (e.g.,  4's  that  look  like  9's). 
Forced  rankings  may  result  in  the  same  subject  rejection  problems  encoun¬ 
tered  in  forced  choice  items.  In  general,  numbered  response  items  are  not 
raeoanmnded.  (Manually,  1967,  p.  521). 
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Differ*! ace  Scales:  Some  items  are  designed  to  provide  a  difference 
or  comparison  score  by  asking  subjects  to  answer  an  item  (a)  as  it  actually 
is  ("actual”)  and  (b)  as  they  would  like  it  to  be  ("ideal ,"  "should  be"). 


trample: 


*1  can  talk  to  my  boss. 


Actual 


Should  be 


The  difference  score  is  provided  by  subtracting  the  "actual"  rating  from 
the  "should  be"  rating,  4-2*2. 

Sometimes  an  importance  scale  is  also  added  to  this  type  of  item. 

example: 

How  important  to  you  is  being  able  to  talk  to  your  boss? 


Of  very 

little 

importance 


Of  very 

great 

importance 


A  single  item  format  which  provides  a  difference  score  is  suggested  by 
Turney  and  Cohen,  1976. 


Example: 


Circle  the  one  appropriate  number  on  the  rating  scale  below  which  best 
describes  your  feeling  about  your  job. 

The  extent  to  which  my  supervisor  pushes  for  increased  productivity 
from  me: 


Just 

right 


Too 

little 


In  this  example  the  difference  score  would  be  equal  to  the  subject's  rating 
subtracted  from  the  midpoint,  number  4. 

Difference  scales  have  the . advantage  of  providing  data  which  suggest 
specific  directions  for  change.  (See  discussion  under  "Intervention  Utility, 
below.) 

Other  Response  Formats:  Some  climate  survey  questionnaires  ask  for 
free  responses  (e.g.,  "What  do  you  like  about  your  job?")  or  specific  data 
(e.g.,  "What  percentage  of  your  time  do  you  spend  working  with  persons  out¬ 
side  your  work  group?") .  Perkins  (1977)  has  used  an  ingenious  method  of 


eliciting  responses  from  illiterate  subjects  (in  this  case,  retarded  chil¬ 
dren,  senile  person,  and  schizophrenic  patients).  He  and  his  associates 
developed  a  large  "speedometer"  with  a  pointer  which  subjects  could  physi¬ 
cally  move  to  indicate  their  response  to  items  read  to  them.'  Similar 
techniques  include  providing  subjects  with  large  Likert-type  scales  and 
asking  them  to  point  to  the  place  on  the  scale  that  best  describes  how 
they  feel,  and  Q-sort  methods  in  which  subjects  indicate  their  responses 
by  choosing  3x5  cards  with  response  alternatives  into  "most  preferred" 
and  "least  preferred"  piles.  Kerr  (1948)  has  described  a  "tear  ballot" 
method  which  asks  respondents  to  tear  the  questionnaire  at  the  response 
they  wish  to  make  (saving  administration  time  and  such  materials  as  pen¬ 
cils  and  answer  forms) .  Small  group  researchers  have  recorded  responses 
to  work  group  processes  by  having  subjects  press  buttons  linked  electroni¬ 
cally  to  recording  and  computing  equipment.  Some  of  these  response  format 
alternatives  may  prove  valuable  in  eliciting  responses  from  Army  personnel 
who  cannot  read  well  enough  to  respond  to  written  surveys.  However,  these 
alternatives  did  not  meet  the  survey  response  format  criteria  and  were  not 
reviewed. 


Scale  Range;  Scales  can  range  from  offering  a  choice  between  only  two 
alternatives  to  the  five intervals  offered  by  the  typical  Likert  scale  or 
the  100  points  offered  in  items  which  ask  respondents  to  estimate  what  per¬ 
centage  they  would  give  a  certain  variable.  In  general,  more  scale  choices 
are  better  because  an  increased  range  of  choices  increases  the  amount  of 
information  or  discrimination  a  respondent  can  provide.  Nunnally  (1967), 
citing  Guilford  (1954)  ,  notes  that  "as  the  number  of  scale  steps  is  in¬ 
creased  from  2  up  through  20,  the  increase  in  reliability  is  very  rapid  .  .  . 
it  tends  to  level  off  at  about  7,  and  after  11  steps,  there  is  little  gain 
in  reliability."  As  a  large  number  of  steps  may  confuse  or  irritate  re¬ 
spondents  and  make  the  survey  instrument  cumbersome  and  difficult  to  score, 
the  optimum  range  for  item  scales  would  appear  to  be  six  to  nine. 

Neutral  Points:  Odd-numbered  scales  have  a  "neutral  point"  which  sub¬ 
jects  can  check  if  they  are  truly  ambivalent  and  cannot  choose  between 
agreeing  or  disagreeing  with  an  item  or  if  they  are  cautious  and  do  not 
want  to  commit  themselves  or  if  they  are  indifferent  to  the  item  and  choose 
a  neutral  response  as  a  way  of  rejecting  the  instrument  (Kaplan,  1972) . 
Even-numbered  scales  force  subjects  to  make  a  choice  one  way  or  the  other. 


Example; 

"I  like  my  job." 

(odd  number-  j _ 1 _ f _ 2 _ k _ 3  [  4 _ i  5 _ I 

5  intervals)  definitely  disagree  so-so  agree  definitely 
disagree  agree 

(even  number-  j _ 1 _ | _ 2 _ j _ 3 _ j _ 4 _ j 

4  intervals)  definitely  disagree  agree  definitely 

disagree  agree 


Nunnally  (1967)  argues  that  the  even-numbered  scale  is  preferable  in  that 
it  elicits  more  variance  in  responses,  especially  among  cautious  respondents 
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whose  response  style  is  to  habitually  select  the  neutral  point.  Gould 
(1976)  and  Perkins  (1977)  caution,  however,  that  even-numbered  scales  may 
cause  rejection  problems  by  making  subjects  feel  uncomfortable,  and  can 
result  in  rectangular  rather  than  normal  response  score  distributions . 

Most  instruments  reviewed  in  the  present  study  used  5-  or  7-point  Likert 
scales  despite  a  preference  in  the  psychometric  literature  for  even- 
numbered  scales  without  neutral  points. 

Item  Content 

Perception  and  attitude  theory  suggests  that  any  work  environment 
questionnaire  item  can  have  one  of  four  components: 

1.  A  cognitive  (or  attribution)  element :  what  a  person  thinks  or 
believes.  Example :  "Supervisors  should  treat  their  subordinates 
fairly." 

2.  An  emotional  (or  affective  evaluation)  element:  what  a  person 
feels.  Examples:  "I  feel  my  supervisor  treats  me  fairly"  or 
"I  like  my  supervisor." 

3.  A  descriptive  component:  what  a  person  objectively  observes. 
Example:  "My  supervisor  treats  subordinates  fairly." 

4.  A  behavioral  component  (a  predisposition  to  act):  what  a  person 
plans  to  do.  Example:  "I  will  file  an  unfair  labor  practices 
complaint." 

This  distinction  goes  to  the  heart  of  what  work  environment  or  organization 
climate  is.  some  researchers,  citing  the  high  correlation  of  affective 
evaluation  "satisfaction"  variables  with  other  dimensions,  argue  that  re¬ 
spondent  perceptions  of  other  climate  work  environment  characteristics  are 
purely  subjective  and  do  not  provide  an  accurate  assessment  of  the  facts 
of  the  organizational  situation  (Guion,  1973).  other  researchers,  citing 
data  that  "descriptive”  variables  do  not  always  correlate  with  "satisfac¬ 
tion,  evaluations,  and  that  these  climate  characteristics  have  been  shown 
to  predict  organizational  outcomes,  argue  that  climate  is  a  viable  work 
environment  construct  (James  6  Jones,  1974;  Schneider,  1975;  Hellriegel  & 
Slocum,  1974) ,  although  most  propose  complex  models  which  include  all  four 
components  discussed  above. 

Researchers  and  practitioners  appear  to  agree  that  items  which  ask 
respondents  to  describe  observable  behavior  or  characteristics  of  super¬ 
visors,  jobs,  or  work  groups  and  organizations  are  more  useful  than  items 
which  ask  for  affective  or  satisfaction  evaluations  (Affourtit,  1976, 
p.  76).  Taylor  and  Bowers  (1972)  note: 

Items  were  not  designed  to  measure  attitudes  of  respondents, 
but  rather  their  perceptions  of  organization  behavior.  A  ques¬ 
tion  of  the  "attitudinal"  type  might  be  phrased,  "How  do  ycu 
feel  about  the  way  your  supervisor  pays  attention  to  what  you 
say?"  The  "descriptive”  type  would  instead  ask,  "How  much  does 
your  supervisor  pay  attention  to  what  you  say?"  This  latter 
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approach  to  measurement  of  perceived  behavior  results  in  data 
which  lend  themselves  more  readily  than  attitude  data  to  sug¬ 
gesting  ways  in  which  change  could  be  attempted,  (page. 2) 


Any  OESO  consultants  (Godina,  1977;  Curry,  1977;  Hinds,  1977;  Sayre, 
1977)  an  unanimous  in  thinking  that  survey  items  and  variables  which  tell 
an  officer  what  he  can  do  about  the  problems  he  has  are  more  useful  than 
evaluative  items  which  assess  satisfaction,  motivation,  and  morale.  "Ques¬ 
tions  which  tell  you  the  troops  aren't  motivated  just  aren't  very  useful 
because  they  don't  say  why  they  aren't  motivated  or  what  a  consultant  or 
leader  should  do  to  motivate  them."  Mikola  (1977)  has  proposed  dropping 
the  evaluative  items  and  variables  in  the  Army's  General  Organizational 
Questionnaire  to  simplify  feedback  procedures  and  focus  on  descriptive 
process  variables  which  suggest  intervention  alternatives. 

A  majority  of  the  instruments  reviewed  include  both  descriptive  and 
evaluative  items  and  did  not  clearly  distinguish  between  them  in  variable 

scales. 

Comprehensibility 

Survey  items  must  be  easy  for  respondents  to  read  and  understand. 
Important  criteria  here  are  reading  level  of  the  vocabulary  and  concepts 
used  in  items,  item  length,  semantics  (do  the  words  make  sense  to  respon¬ 
dents)  ,  syntax  (do  the  sentences  make  sense) ,  unidj  mens ion ality  (does  each 
item  ask  for  only  one  judgment) ,  and  appearance. 

Heading  level;  An  eighth  grade  reading  level  is  usually  recommended 
for  survey  items.  Vocabulary  at  this  level  is  sufficient  to  express  nearly 
all  work  environment  or  organisational  climate  questionnaire  concepts  while 
remaining  comprehensible  to  most  lower  level  enlisted  personnel.  (The 
literacy  of  recruits  in  the  all-volunteer  military,  initially  seen  as  a 
possible  obstacle  to  the  use  of  surveys  in  organisational  effectiveness 
efforts,  has  apparently  not  proven  to  be  a  problem.)  When  in  doubt,  the 
rule  is  "simpler  is  better." 

Item  length;  Shorter  is  better.  long  item  tend  to  confuse  subjects 
and  are  more  likely  to  contain  norm  than  one  factor  (see  "unidimension¬ 
ality"  below) .  Eight  to  15  words  is  considered  optimum  for  an  item. 

Semantics;  Items  should  use  words  and  phrases  that  are  as  closa  as 
posslbl#  to  the  waya  respondents  actually  talk.  Tor  example ,  Affourtit 
(1977)  found  in  work  with  the  Narine  Corps  that  recruits  did  not  understand 
questions  which  referred  to  "my  work  group  supervisor"  but  immediately  un¬ 
derstood  the  sane  questions  if  the  supervisor  was  refarred  to  as  "my  NCO. " 
The  latter  language  was  "Narine"  while  the  former  was  in  organizational 
terms.  The  rule  again  is  to  keep  it  simple  and  to  talk  military;  don't 
talk  behavioral  science  "jargon"  in  writing  survey  items. 

ffyntax;  Once  again,  simple  and  clear  is  best.  Tortured  syntax  (long 
sentences  with  many  modifying  clauses)  confuses  and  irritates  respondents. 
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Appearances  To  minimize  confusion  and  errors  in  recording  and  analyz¬ 
ing  data,  and  to  motivate  participants,  survey  questionnaires  should  be  as 
clear  and  attractively  formatted  as  possible.  Specific  suggestions  include 
use  of  large,  clear  print;  logical  organizational  and  sequencing  of  items 
(only  one  column  of  items  on  a  page,  for  example) ;  and  as  much  "white 
space"  as  possible.  Babbie  (1973)  notes: 

The  format  of  a  questionnaire  can  be  just  as  important  as  the  na¬ 
ture  and  wording  of  the  questions  asked.  An  improperly  laid  out 
questionnaire  can  lead  respondents  to  miss  questions,  can  confuse 
them  as  to  the  nature  of  the  data  desired,  and,  in  the  extreme, 
can  lead  to  respondents  throwing  the  questionnaire  away. 

As  a  general  rule,  the  questionnaire  should  be  spread  out  and  un¬ 
cluttered.  The  researcher  should  maximize  the  "white  space"  in 
his  instrument .  Inexperienced  researchers  tend  to  fear  their 
questionnaire  will  look  too  long,  and  as  a  result,  they  squeeze 
several  questions  on  a  single  line,  abbreviate  questions,  and  try 
to  use  as  few  pages  as  possible.  All  these  efforts  are  ill- 
advised  and  even  dangerous.  Putting  more  than  one  question  on  a 
line  will  result  in  some  respondents  skipping  the  second  question. 
Abbreviating  questions  will  result  in  misinterpretations.  And 
more  generally,  the  respondent  who  finds  he  has  spent  consider¬ 
able  time  on  the  first  page  of  what  seemed  a  short  questionnaire 
will  be  more  demoralized  than  the  respondent  who  quickly  com¬ 
pleted  the  first  several  pages  of  what  initially  seemed  rather 
long.  Moreover,  the  second  respondent  will  have  made  fewer  er¬ 
rors  and  will  not  have  been  forced  to  reread  confusing,  abbrevi¬ 
ated  questions. 

The  desirability  of  spreading  questions  out  in  the  questionnaire 
cannot  be  overemphasized.  Squeezed-together  questionnaires  are 
disastrous,  (p.  45) 

The  format  in  which  subjects  indicate  their  responses  can  also  be  im¬ 
portant.  Response  format  alternatives  include: 

(1)  checking  a  point  on  a  line 


(2)  circling  a  number 

1  —0—  3  —  4  —  5  —  6 
True  00 

False  rn 


(3)  marking  a  box 


or: 


(4)  patting  a  number  in  a  box 


Ztem  answer  [T] 

(5)  marking  a  space  on  a  computer  scorable  answer  sheet 

II  II  II  II  II  | 

1  2  3  4  5  6 

For  ease  in  scoring  and  processing,  the  computer  sheet  alternative  is 
best,  although  subjects  unfamiliar  with  this  format  must  be  carefully  in¬ 
structed  in  how  to  transfer  their  scores  from  the  instrument  item  booklet 
to  the  answer  sheet.  Otherwise,  the  important  consideration  is  that  the 
response  format  be  large  and  clear  enough  so  that  subjects  are  not  confused 
and  their  marks  are  easily  readable.  "Bunched  up"  and  multiple  line  scales 
cause  problems. 


Ex/unpla: 


(a)  Bunched  up 


_s_X_s— s— !— 


(b)  Multiple  line 


□  □ 

1  2 

□  □ 

3  4 

For  this  reason,  if  computer  score  sheets  are  not  used,  alternatives  (3) 
(circle  a  number)  and  (4)  (mark  a  box)  are  best. 

Vnidimanaionali ty:  Survey  items  should  ask  only  one  question,  that  is 
they  should  ask  respondents  to  make  a  judgment  about  only  one  factor  in  a 
given  item.  For  example,  the  statement,  "My  supervisor  is  friendly  and 
easy  to  approach,  and  sets  high  standards  for  his  work  group"  implies  two 
questions i  (1)  Is  he  friendly  and  easy  to  approach?  and  (2)  Does  he  set 
high  standards?  A  respondent  with  only  one  rating  scale  to  respond  to  an 
item  of  this  type  may  rate  one  part  of  the  question  (giving  either  no  data 
or  inaccurate  data  about  the  other  part  of  the  question) ,  give  a, compromise 
answer  which  provides  inaccurate  data  on  both  factors,  or  be  confused  and 
refuse  to  answer.  Even  modifying  clauses  are  suspect.  Another  item,  "We 
spend  a  lot  of  time  sitting  around  doing  nothing,  without  any  meaningful 
work  to  do"  may  ask  a  question  about  both  "sitting  around  doing  nothing" 
and  "meaningful  work." 

the  following  example ,  encountered  in  the  instrument  review,  illus¬ 
trates  things  to  avoid  in  item  construction: 

To  what  extent  have  the  consultants  produced  an  environment  that 

encourages  continual  learning,  training,  and  active  interest 


regarding  the  job,  and  the  product  of  service  to  which  the  job 
contributes  a  setup  that  enables  an  employee  to  use  and  develop 
his  personal  skills  and  knowledge,  which  in  turn  affects  his 
environment  and  self-esteem  obtained  from  the  work  itself? 

This  item  includes  no  fewer  than  eight  different  factors;  its  vocabulary, 
length,  semantics,  and  syntax  make  it  difficult  to  understand. 

Items  can  be  pilot-tested  for  comprehensibility  and  unidimensionality 
in  a  number  of  ways.  A  rigorous  approach  (Thurston  £  Chave,  1929)  is  to 
have  a  number  of  subjects  rate  each  item  from  1  to  11  on  the  basis  of  how 
well  it  expresses  the  question  it  asks  (e.g.,  how  "positive"  or  "negative" 
it  sounds) .  Items  to  which  most  subjects  give  approximately  the  same  rat¬ 
ing  are  considered  "good"  (understandable  and  unambiguous)  and  are  kept 
for  use  in  the  survey  being  constructed.  Items  about  which  the  sample 
subjects  disagree  by  giving  widely  varying  ratings  are  considered  "bad" 
(incomprehensible ,  multidimensional,  or  ambiguous)  and  jure  discarded.  A 
simpler,  less  time-consuming  approach  is  to  take  a  "lowest  conanon  denomi¬ 
nator"  respondent  and  try  items  out  on  him.  Affourtit  (1977)  first 
screened  all  items  used  in  a  Marine  Corps  survey  by  reading  them  to  a 
brain-damaged  Marine  recruit  with  a  tested  I.Q.  of  7B.  Any  items  that 
this  person  could  not  understand  and  respond  to  were  discarded. 

Items  in  the  instruments  reviewed  in  the  present  study  varied  widely 
in  comprehensibility.  In  general,  formal  theory-based  items  tend  to  be 
overly  complex  and  to  use  behavioral  science  jargon,  where  grounded  theory- 
based  instruments  were  more  readable. 

Response  bias;  The  ways  items  are  constructed  can  cause  "response 
bias":  distortion  or  error  in  subject  judgments  about  job  or  organiza¬ 
tional  environments  due  to  the  nature  of  the  survey.  Response  bias  con¬ 
siderations  include  response  set,  response  styles,  and  instrument  rejection 

Response  set:  This  refers  to  the  format,  and  consistency  or  inconsis¬ 
tency  of  the  format,  in  which  items  are  presented.  If  items  are  always 
presented  in  the  same  way  (i.e.,  response  set  is  consiste-.t) ,  respondents 
can  get  in  the  habit  of  responding  to  them  in  a  certain  way  without  really 
reading  them. 

Consistent  positive  response  set  is  a  particular  problem  with  Likert 
scales  which  have  a  consistent  positive  response  bias;  that  is,  where  "1" 
is  always  "bad"  and  "5"  is  always  "good.” 

Example:  "My  supervisor  is  friendly  and  easy  to  approach" 

1 _ 2 _ 3 _ 4 _ 5 

to  a  to  a  some-  to  a  to  a 

very  small  what  great  very 

small  extent  extent  great 

extent  extent 


If  an  entire  instrument  is  constructed  in  this  fashion,  respondents  may  get 
in  the  habit  of  agreeing  with  items  and  checking  "4sM  without  really  trying 
to  make  accurate  decisions  about  these  items.  Peabody  (1964)  cites  data 
to  show  that  60-70  percent  of  the  variance  in  responses  can  be  accounted 
for  by  the  response  set.  The  usual  method  of  minimizing  response  set  bias 
is  to  vary  the  set  by  alternating  positive  and  negative  items  in  the  sur¬ 
vey  questionnaire  and  using  a  semantic  differential  format  which  presents 
a  visually  balanced  stimulus. 

Example: 


1  - 2  -  3  -  4 

My  supervisor 
is  not  friendly 
and  easy  to 
approach 


5  -  6 

My  supervisor 
is  friendly 
and  easy  to 
approach 


1  -  2  -  3 

This  organiza¬ 
tion  does  not 
have  a  lot  of 
red  tape  which 
gets  in  the  way 
of  getting  the 
job  done 


-  4  -  5  -  6 

This  organiza¬ 
tion  has  a  lot 
of  red  tape 
which  gets  in 
the  way  of 
getting  the 
job  done 


In  the  first  example,  the  "good"  response  is  on  the  right,  but  in  the 
second  exanqple  it  is  on  the  left.  The  use  of  reversed  or  negative  items 
may,  however,  introduce  other  sources  of  error.  For  exanqple,  readers  used 
to  positive  items  may  skip  over  the  "nots"  in  reversed  items  and  respond 
as  though  these  items  were  positive  (Babbie,  1973,  p.  144).  Negative  terms 
such  as  unfriendly  or  fail,  may  have  an  emotional  impact  which  actually 
causes  subjects  to  bias  their  responses  in  a  positive  direction  (Taylor  & 
Bowers,  1972,  p.  24).  Semantic  differential  format  may  eliminate  these 
sources  of  error,  but  this  question  should  be  further  researched. 

Roughly  two-thirds  of  the  instruments  reviewed  in  the  present  study 
varied  positive  and  negative  items  to  avoid  the  problems  of  a  consistent 
response  set;  one-third  used  a  consistent  positive  response  set. 

Item  order  and  grouping  refers  to  second  response  set  bias  which  may 
be  introduced  by  gathering  all  the  items  relating  to  a  certain  topic 
(e.g. ,  supervisor  behavior)  together  in  the  survey. 

Example: 

(1)  "My  supervisor  is  friendly  and  easy  to  approach.” 

(2)  "My  supervisor  sets  high  standards  for  his  work  group." 


If  a  subject  answers  positively  or  negatively  to  questions  about  supervisor 
friendliness,  perceptions  on  this  dimension  may  spill  over  to  bias  responses 


•bout  the  quit*  different  sacond  factor,  standards.  The  tendency  for  re¬ 
spondents  to  answer  subsequent  questions  in  a  manner  consistent  with  their 
assessment  of  preceding  items  is  sometimes  called  the  "halo”  effect  (Moos, 
1974) . 


This  problem  is  usually  dealt  with  by  introducing  distr actor  items 
which  have  nothing  to  do  with  the  variable  really  being  measured  (e.g., 

“I  like  the  climate  in  the  part  of  the  country  in  which  1  am  stationed”) 
or  by  listing  items  in  random  order  so  that  items  relating  to  the  same 
variable  do  not  systematically  appear  together.  The  latter  method  ap¬ 
pears  to  be  preferable,  as  no  items  are  wasted  and  questions  about  differ¬ 
ent  variables  act  as  adequate  distractors  for  one  another .  Babbie  (1973) 
notes,  however,  that 

A  "randomized”  set  of  questions  trill  probably  strike  the  respon¬ 
dent  as  chaotic  and  worthless.  It  will  be  difficult  for  him  to 
answer,  moreover,  since  he  must  continually  switch  his  attention 
from  one  topic  to  another,  and  finally,  even  a  randomized  order¬ 
ing  of  questions  will  have  the  [consistent  response  set  bias] 
effect  discussed  above— except  that  the  researcher  will  have  no 
control  over  the  effects,  (p.  148) 

Most  instruments  reviewed  in  the  present  study  presented  items  in  random 
order. 


Other  item  order  considerations  include  the  degree  of  interest  or  threat 
in  items  placed  first  in  a  questionnaire.  Some  authors  advocate  putting 
more  interesting,  personally  relevant,  or  salient  items  first  to  motivate 
participants  to  complete  the  survey: 

Zt  is  usually  best  to  begin  the  questionnaire  with  the  most  in¬ 
teresting  set  of  questions.  The  potential  respondent  who  glances 
casually  over  the  first  few  questions  should  want  to  answer  them. 

Perhaps  they  will  ask  for  attitudes  that  he  is  aching  to  express. 

At  the  same  time,  however,  the  initial  questions  should  not  be 
threatening.  (Zt  might  be  a  bad  idea  to  begin  with  questions 
about  sexual  behavior  or  drug  abuse.)  Requests  for  duller  demo¬ 
graphic  data  (age,  sex,  and  the  like)  should  generally  be  placed 
at  the  end  of  the  self-administered  questionnaire.  Placing  these 
questions  at  the  beginning,  as  many  inexperienced  researchers  are 
tempted  to  do,  gives  the  questionnaire  the  initial  appearance  of 
a  routine  form,  and  the  person  receiving  it  may  not  be  motivated 
to  complete  it.  (Babbie,  1973,  p.  150) 

Sudman  and  Bradbum  (1974)  cite  data  to  recommend  the  opposite  approach, 
placing  less  threatening  questions,  such  as  those  concerning  demographic 
data,  early  in  the  survey  to  give  respondents  time  to  warm  up  and  learn 
how  to  respond  to  items.  Affourtit  (1977)  suggests  immediately  establish¬ 
ing  the  salience  of  the  survey  by  asking  military  officers  to  report  their 
units'  results  on  "hard"  command  performance  indicators  such  as  inspection 
scores,  retention  rates,  and  incidence  of  racial  conflicts.  Most  of  the 
instruments  reviewed  placed  demographic  questions  (age,  sex,  rank,  mili¬ 
tary  occupation  specialty,  unit  identification  number)  first. 
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gasgonse  styles .  Error  in  survey  variable  measures  can  also  be  caused 
by  tendencies  of  individual  subjects  to  respond  in  consistently  biased  ways, 
lbs  sost  important  of  these  response  styles,  which  to  sobs  extent  can  be 
avoided  or  corrected  by  careful  item  construction,  are  acquiescence ,  social 
desirability,  understatement  (caution),  overstatement,  deviance,  and  incon¬ 
sistency  "lie.” 

Acquiescence,  or  agreement  tendency,  refers  to  the  tendency  of  sub¬ 
jects  to  agree  with  each  item  (e.g.,  to  answer  "true”  more  often  than 
"false”  on  true-false  tests,  or  to  usually  check  the  extreme  "good"  end  of 
Likert  scales) ,  especially  if  there  is  a  consistent  response  set.  There 
is  considerable  debate  in  the  measurement  literature  about  the  importance 
of  this  factor,  although  Nunnally  (1967)  concludes: 

The  overwhelming  weight  of  the  evidence  now  points  to  the  fact 
that  the  agreement  tendency  is  of  very  little  importance  either 
as  a  measure  of  personality  or  as  a  source  of  systematic  in¬ 
validity  in  measures  of  personality  and  sentiments.  What  little 
stylistic  variance  there  is  came  of  that  tendency,  if  any,  can 
be  mostly  eliminated  by  ensuring  that  an  instrument  is  con¬ 
structed  so  that  there  is  a  balance  of  items  keyed  "agree”  and 
"disagree"  with  respect  to  the  trait  in  question,  (p.  612) 

In  other  words,  the  acquiescence  problem  can  be  avoided  by  having  both 
positive  and  negative  items  so  that  there  is  not  a  consistent  positive 

response  set. 

Social  Da* ir ability  refers  to  the  tendency  of  respondents  to  say  good 
things  about  themselves  and  to  agree  with  "everyone"  in  the  culture  or  or¬ 
ganisation  as  to  what  is  good  or  desirable,  whether  or  not  they  really  agree. 

Example: 


T&is  organiza¬ 
tion  should 
have  high 
standards 

Few  respondents  are  likely  to  agree  with  the  idea  that  their  organisa¬ 
tion  should  not  have  high  standards,  even  if  they  feel  pressured  by  the 
standards  that  exist.  Similarly,  items  which  ask  for  the  degree  of  a  re¬ 
spondent's  agreement  with  organizational  policy  statements  or  particularly 
prestigious  (or  disliked)  figures  are  likely  to  result  in  biased  responses. 

Example; 

"Do  you  agree  with  the  Commanding  General  that  base  discipline  is 
too  lax?" 

"Do  you  agree  with  Socialist  Worker  Party  demonstrators  that  sol¬ 
diers  should  not  be  allowed  in  town?" 


This  organisa< 
tion  should 
not  have  high 
standards 
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Social  desirability  problems  can  be  minimized  by  making  items  as  neu¬ 
tral  and  descriptive  as  possible,  by  balancing  the  alternatives  in  tone  so 
that  anchors  sound  equally  reasonable  and  socially  desirable,  and  by  dis¬ 
tancing  questions  so  subjects  are  not  asked  to  give  sensitive,  socially 
undesirable  data  about  themselves. 

Example: 

1  -  2  -  3  -  4  -  5  -  6 

Higher  stand-  Higher  stand¬ 
ards  are  not  ards  are 

needed  in  this  needed  in  this 

organization  organization 

This  item  asks  the  "standards”  question  in  a  somewhat  less  biased  way: 
A  subject  could  check  the  low  end  of  the  scale,  indicating  that  higher 
standards  are  not  needed,  either  because  the  organization's  standards  are 
already  high  or  he  feels  pressured  by  existing  standards. 

Similarly,  in  asking  about  drug  use  in  a  unit,  an  item  like  "I  smoke 
a  lot  of  marijuana”  is  less  likely  to  elicit  an  accurate  response  than 
"People  in  this  work  group  smoke  marijuana.”  The  latter  item  distances 
the  question  by  asking  the  subject  about  others'  behavior  rather  than  his 
own.  Perkins  (1977)  reports  that  the  use  of  the  first  person  pronoun  "I" 
changes  the  response  set  enough  to  elicit  significantly  different  responses 
to  otherwise  identical  questions. 

Social  desirability  response  (and  "lie")  tendencies  of  subjects  can  be 
identified  by  including  in  a  survey  such  items  as,  "I  clean  my  rifle  every 
day."  Such  performance  sounds  good,  but  few  subjects  actually  do  it.  Data 
from  subjects  who  consistently  score  high  on  such  items  are  suspect.  Social 
desirability  biases  are  often  estimated  by  administering  the  Crowne-Marlowe 
Social  Desirability  Scale  with  the  instrument  being  tested,  instrument  items 
correlating  highly  with  the  Crowne-Marlowe  Scale  are  discarded. 

Understatement  or  cautious  response  tendencies  emerge  with  respondents 
who  habitually  avoid  taking  a  stand  on  items  by  checking  neutral  points, 
points  near  the  middle  of  scale,  or  the  least  extreme  anchor  alternatives 
on  scales.  This  response  bias  can  be  avoided  to  some  extent  by  using  even- 
number  interval,  no-neutral  point,  or  dichotomous  (true-false)  scales  which 
force  people  to  take  a  stand.  Cautious  responses  can  be  corrected  for  in 
statistical  analysis  of  survey  data  by  identifying  data  with  significantly 
smaller  item  variance  than  found  for  the  sample  as  a  whole  and  inflating 
the  understated  data  to  average  levels.  More  simply,  markedly  deviant  data 
can  be  discarded. 

Kxtrama  or  deviant  responses  appear  with  respondents  who  deliberately 
check  the  extreme  positive  or  negative  ends  of  scales,  or  differ  radically 
from  the  average  response  given  by  most  subjects.  These  responses  can  be 
limited  by  reducing  the  number  of  scale  intervals  or  by  using  dichotomous 
scales  where  extreme  responses  are  impossible,  although  both  methods  sacri¬ 
fice  reliability  (Nunnally,  1967,  p.  613).  Extreme  responses  can  also  be 
identified  and  adjusted  or  discarded  in  statistical  analysis  of  survey 
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data.  In  general,  items  which  alicit  aithar  cautious  or  extreM  responses 
(i.e.,  show  little  variance  or  axtraM  variance)  from  a  Majority  of  sub¬ 
jects  should  ha  discarded,  because  in  this  case  the  fault  is  probably  in 
the  item  rather  than  the  respondent. 

inconsistency  ("lie")  responses  are  those  in  which  subjects  differ 
markedly  in  their  responses  to  very  siailar  items  Masuring  the  sass  variable 
(for  example,  rating  a  supervisor  high  on  "friendliness"  on  one  item,  and 
low  on  "acting  in  a  friendly  way"  on  a  subsequent  item) .  This  type  of  in¬ 
consistency  can  be  due  to  deliberate  lying  or  rejection  of  the  instrument, 
or  to  confusion  or  lack  of  con?) rehens ion.  inconsistency  is  easily  identi¬ 
fied  in  statistical  analysis  of  instrument  data  (by  comparison  of  responses 
to  similar  items,  or  unusual  variance  in  items  Masuring  the  saae  variable) 
and  can  be  adjusted  by  discarding  deviant  responses  or  all  data  for  the 
subject.  Again,  if  many  subjects  report  inconsistent  data,  the  problem 
is  probably  the  instrument,  and  it  should  be  redesigned. 

Only  one  of  the  instruments  reviewed  attempted  to  identify  response 
style  biases.  Moos  (1974)  constructed  an  extreme  response  "halo  effect" 
and  an  inconsistency  scale  of  five  extremely  positive  and  five  extreMly 
negative  items  to  which  fewer  than  10  percent  of  his  sample  population  re¬ 
sponded.  Subjects  who  did  respond  to  these  items  were  presumed  to  be  ly¬ 
ing,  responding  extreMly,  or  inconsistently,  if  their  responses  were  di¬ 
vergent.  In  the  final  version  of  his  instrument,  however.  Moos  dropped 
this  subscale  because  he  found  that  data  from  subjects  who  scored  high  on 
it  were  unusable  and  would  be  discarded  for  other  reasons  (rejection  of 
the  instrument  or  failure  to  complete) .  Few  work  environment  or  organiza¬ 
tional  climate  researchers  report  considering  or  designing  instruments  to 
minimize  response  bias.  (See  page  110  for  Addendum.) 


Item  and  Instrument  Rejection 
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The  "bottom  line"  of  item  and  instrunent  construction  is  whether  re¬ 
spondents  are  willing  to  complete  the  survey  instrument.  InstruMnt  rejec¬ 
tion  can  taka  several  forms:  refusal  to  fill  out  the  survey  at  all,  omis¬ 
sion  of  specific  items,  or  failure  to  complete  a  questionnaire.  Respondents 
can  also  answer  randomly  or  in  a  certain  pattern:  all  "l's,"  all  "5's,” 
all  neutral  points,  or  such  repeated  patterns  as  "1-2-3-4-5."  Rejection 
can  be  due  to  the  instrument  itself  or  a  variety  of  factors  such  as  subject 
fatigue  or  anger  about  situations  that  have  nothing  to  do  with  the  survey; 
issues  such  as  confidentiality;  or  failure  to  motivate  or  reassure  subjects 
during  the  survey  administration  instructions .  Instrument  factors  most 
likely  to  cause  rejection  are  length,  lack  of  interest,  incomprehensibility , 


and  threat. 


People  get  tired  and  quit  filling  out  questionnaires  which  are  too 
lengthy.  The  number  of  items  in  the  instnsnents  reviewed  averaged  173  and 
ranged  from  29  to  more  than  700.  Estimates  of  the  amount  of  time  needed 
to  complete  depended  on  the  population  being  sampled  and  ranged  from  10  to 
15  seconds  (executive  and  Navy  officers)  to  1  minute  per  item  (Marine  Re¬ 
cruits)  ,  with  a  conservative  time  of  20  seconds  per  item.  Most  consultant 
practitioners,  and  especially  OESOs  (Hinds,  1977;  Sayre,  1977)  felt  the 
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surveys  they  had  used  wax*  too  long  and  conaidarad  100  1 
to  ha  tha  outar  limit  for  survey  length. 


and  1  hour 


Lac*  of  interest  or  saiience  is  another  causa  for  instrument  rejection 
People  get  bored  when  they  do  not  perceive  surveys  to  be  relevant  to  their 
concerns  or  do  not  understand  why  they  are  taking  these  surveys,  and  thus 
tend  to  reject  items  that  do  not  nake  intuitive  sense  to  them  (i.e. ,  lack 
of  "face  validity") .  Xt  follows  that  the  grounded  theory  approach  of 
identifying  the  concerns  or  an  organization's  members  and  then  "tailoring" 
a  survey  to  reflect  these  concerns  in  the  members'  own  language  would  amxi- 
mize  salience  and  face  validity.  Some  authors  suggest  a  movement  away  from 
universal  instruments  applicable  to  all  organizational  settings  toward  the 
development  of  contingent  surveys  applicable  to  specific  people  and  organi¬ 
zations  (Hellriegel  &  Slocum,  1974;  Schneider,  1975).  Administration  in¬ 
structions  which  stress  the  "what's  in  it  for  you"  benefits  to  respondents 
can  significantly  increase  motivation  and  lower  rejection  rates.  (Godina, 
1977) 


Xtems  which  are  incomprehensible  cause  people  to  get  confused,  care¬ 
less,  or  irritated  and  therefore  quit,  for  reasons  discussed  above. 

A  survey  can  appear  threatening  if  people  feel  the  information  they 
are  asked  to  provide  will  prove  harmful  to  themselves  or  others.  This  does 
not  appear  to  be  a  serious  constraint  to  tha  use  of  work  environment  ques¬ 
tionnaires  if  administration  instructions  stress  confidentiality  and  the 
beneficial  ways  in  which  data  collected  will  be  used. 

Unfortunately,  little  data  on  rejection  rates  or  causes  was  available 
on  most  of  the  instruments  reviewed.  Moos  (1974)  found  rejection  rates 
ranged  from  55  to  7  percent  with  a  mean  of  25  percent,  among  hospitalized 
mental  patients  (the  55  percent  rejection  rete  was  with  very  chronic 
schizophrenic  patients).  The  Army's  General  Organization  Questionnaire 
was  initially  rejected  by  15  percent  of  respondents  at  one  installation, 
a  rate  reduced  to  3  percent  by  improved  administration  instructions  (Godina 
1977) .  Affourtit  (1977)  reports  a  rejection  rate  of  less  than  1  percent 
for  the  Marine  Corps  "LEAP"  survey.  Rejection  rates  of  1  to  3  percent  seem 
a  reasonable  objective  for  work  environment  and  organisational  climate 
questionnaires  with  normal  subject  populations. 

Reliability  and  Validity 

The  American  Psychological  Association's  "Standards  for  Educational 
and  Psychological  Tests"  (1974)  states  that  presentation  of  "evidence  of 
reliability  and  validity”  (is)  essential  for  developing  "any  systematic 
basis  for  making  inferences  about  characteristics  of  people . "  Taylor  and 
Bowers  (1972)  argue  that  these  standards  should  be  applied  to  work  environ¬ 
ment  questionnaires  regarding  any  instrument  "used  in  the  making  of  de¬ 
cisions  which  ultimately  may  drastically  affect  the  welfare  of  persons 
tested,  both  in  individual  cases  and  in  the  aggregate"  (p.  81).  Nearly 
all  researchers  who  have  c cemented  on  the  o :c anizational  climate  litera¬ 
ture  criticize  the  field  for  its  low  standards  and  lack  of  reliability  and 
validity  (Guion,  1973}  Taylor  &  Bowers,  1972}  Hellriegel  &  Slocum,  1974; 
Jmaes  fi  Jones,  1974;  Schneider,  1975). 


The  present  study  attempted  to  assess  the  reliability  and  validity  of 
all  instruments  for  which  data  were  available.  Tables  4  and  5  summarize 
these  data.  Xn  the  following  discussion,  it  should  be  emphasised  that 
"not  reported"  does  not  necessarily  naan  "does  not  exist."  Aany  of  the 
findings  reported  here  are  frost  unpublished  sources,  obscure  technical  re¬ 
ports,  or  telephone  conversations  with  researchers.  While  it  is  the  au¬ 
thors  '  impression  that  most  work  environment  questionnaires  lade  relia¬ 
bility  and  validity  statistics,  data  they  were  unable  to  find  may  exist. 

Reliability 

Reliability  describes  an  instrument's  accuracy  or  consistency:  the 
degree  to  which  it  will  give  the  same  reading  whan  used  to  measure  the 
seam  thing  sure  than  once. 

Three  types  of  reliability  are  commonly  cited:  internal  consistency 
(or  split-half)  reliability,  test-retest  (timei  -  tima2)  reliability,  and 
scale  reliability. 

Internal  consistency  reliability  means  that  items  which  purport  to 
measure  the  same  variable  in  fact  agree  or  correlate  with  one  another.  This 
is  tested  statistically  by  dividing  the  instrument  (or  a  set  of  items  mea¬ 
suring  the  same  variable)  into  two  parts  (the  split-half)  and  finding  the 
correlation  coefficient  between  responses  on  items  in  the  two  parts. 

Tm st-retest  reliability  means  that  persons  taking  the  test  at  two 
points  in  tine  will  score  approximately  the  same,  assuming  nothing  has  oc¬ 
curred  during  the  time  between  the  two  tests  which  would  cause  people  to 
respond  differently. 

Stale  reliability  means  that  the  items  which  measure  a  given  variable 
correlate  with  each  other  and  with  the  total  score  for  the  variable.  Cor¬ 
relations  aanng  individual  items  tend  to  be  fairly  low  (r  of  .20  is  con¬ 
sidered  good)  but  item-total  correlations  are  much  higher.  Scale  relia¬ 
bilities  can  be  increased  by  using  items  with  more  interval  steps,  and  by 
using  more  items.  For  example,  30  dichotomous  items  may  be  needed  to 
achieve  a  scale  reliability  of  .80,  but  only  15  seven-point  Likert  items 
would  be  needed  to  attain  the  same  level  of  reliability  (Nunnally,  1967, 
p.  265) .  These  standards  may  be  impractical  for  work  environment  question¬ 
naires,  which  tend  to  have  only  three  to  six  items  per  variable. 

Validity 

Validity  describes  the  extent  to  which  an  instrument  measures  some¬ 
thing  real,  an  assessment  made  by  comparing  its  results  to  the  results  of 
other  measures  or  to  outcome  criteria  such  as  inspection  scores  and  reten¬ 
tion  rates.  Several  types  of  validity  are  referred  to  in  the  measurement 
literature:  face  validity,  content  validity,  construct  validity,  conver¬ 
gent  validity,  criterion  validity,  concurrent  validity,  and  predictive 
validity. 

Fees  validity  means  that  the  instrument  sounds  right  or  makes  intui¬ 
tive  sense  to  users,  that  it  uses  the  language  and  concepts  of  the  sample 
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population  with  which  it  will  be  used.  This  quality  has  been  referred  to 
a a  salience  in  the  discussion  of  item  construction .  For  example ,  command¬ 
ing  officers'  color  preference  might  prove  to  be  a  good  predictor  of  in¬ 
spection  scores,  but  no  consultant  could  seriously  propose  to  use  this 
measure  in  a  military  organisation  because  of  its  lack  of  face  validity. 
Face  validity  has  no  scientific  meaning,  but  is  a  critically  important 
practical  factor  in  gaining  acceptance  for  any  measure  to  be  used  in  or¬ 
ganisational  effectiveness.  It  can  be  tested  statistically,  to  a  degree, 
by  using  the  Thurstone  rating  method  described  earlier:  Sample  subjects 
rate  each  item  on  face  validity  on  a  two-point  scale  and  items  about  which 
there  is  no  substantial  disagreement  are  discarded. 

Content  validity  means  that  the  instrument  variables  measure  what  they 
say  they  measure  (e.g.,  that  a  variable  labeled  "standards"  in  fact  asks 
questions  such  as,  "To  what  extent  does  your  supervisor  set  high  standards 
for  your  work  performance?") .  This  would  seem  an  obvious  point,  but  the 
instrument  review  found  a  surprising  number  of  cases  in  which  items  in¬ 
cluded  in  variable  factors  had  virtually  no  relation  to  variables  they 
presumably  measured.  Nunnally  (1967)  notes  that  "the  two  major  standards 
for  insuring  content  validity  are  (1)  a  pre-presentative  collection  of 
items  and  (2)  'sensible*  methods  of  test  construction"  (p.  81).  in  other 
words,  common  sense.  Content  validity  can  be  estimated  statistically,  to 
some  degree,  by  the  Thurstone  method  (asking  people) ,  by  examination  of 
internal  inconsistency  reliabilities  (on  the  assumption  that  if  items  cor¬ 
relate,  they  are  probably  measuring  the  same  thing) ,  and  comparison  with 
other  measures  of  the  same  variable  dimension,  a  topic  to  be  discussed 
under  construct  and  convergent  validity. 

Construct  validity  means  that  there  is  some  independent  evidence  to 
confirm  that  the  concept  of  construct  measured  by  an  instrument's  variables 
really  exists.  This  is  important  because  behavioral  science  constructs 
such  as  "ego  state”  (psychology)  or  "team  spirit”  (organizational  behavior) 
are  hard  to  define  and  measure  and  hence  are  controversial.  Guion  (1973) 
argues,  for  example,  that  the  whole  construct  of  work  environment  or  or¬ 
ganisational  climate  is  meaningless . 

Construct  validity  is  usually  established  by  correlating  one  measure 
of  a  variable  with  another  amasure  of  the  same  variable.  For  example , 
Patchen  (1965)  established  construct  validity  by  comparing  subordinates' 
rating  of  team  spirit  on  a  work  environment  questionnaire  with  the  number 
of  cars  displaying  the  organization's  bumper  sticker  in  the  company  park¬ 
ing  lot.  The  assumption  in  this  case  was  that  voluntary  display  of  the 
corporate  insignia  was  an  indication  of  pride  in  belonging  to  the  organi¬ 
sation.  Survey  instrument  scores  can  similarly  be  correlated  with  the 
results  of  interviews,  other  surveys,  group  sensing  sessions,  or  outcome 
measures  of  comparable  variables.  This  validation  procedure  is  sometimes 
called  convergent  validity:  If  several  independent  measures  converge, 
giving  approximately  the  same  score  on  a  variable,  they  establish  its 
construct  or  convergent  validity. 

Ctitarion  validity  means  that  a  variable  measure  correlates  with  some 
"hard”  outcome  or  criteria  (for  example,  standards  as  measured  by  an  or¬ 
ganizational  climate  questionnaire  with  inspection  scores) .  Criterion 
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validity  can  ba  aatabliahad  at  two  point*  in  tin*  s  at  the  um  tin*  the 
outcome  is  Mtnnd,  in  which  case  it  is  called  concurrent  validity,  or 
by  predicting  a  future  out cone,  called  predictive  validity. 

Concurrent  validity  is  the  moat  commonly  cited  validity  neasure  for 
work  environment  and  organisation  climate  surveys  because  it  is  easier  to 
get  the  data  required  for  concurrent  analyses.  The  three  moat  rrsunn  de¬ 
signs  encountered  in  this  review  were  (1)  correlation  of  instrument  process 
indices  with  out cos*  perceptions  nsasured  by  the  sane  instrument;  (2)  cor¬ 
relation  of  survey  indices  with  the  results  of  several  coaqparable  organi¬ 
sations  (different  plants  Baking  the  ease  product,  or  destroyers  with  ap- 
proxinately  the  saae  crews  and  missions)  on  sane  outcome  variable;  and 
(3)  "split-half  designs,  in  which  survey  variable  data  on  the  best  and 
worst  performing  organisations  (units  ranked  in  the  top  and  bottom  10  per¬ 
cent  on  inspection  scores)  were  compared,  (it  should  be  noted  that  the 
first  approach  is  not  really  a  criterion  validity  design  because  the  in¬ 
strument  indices  are  simply  correlated  with  one  another,  not  with  an  ob¬ 
jectively  observable  outcome  in  the  real  world.) 

Prmdlctlvm  validity  designs  provide  much  mors  stringent  tests  of  in¬ 
strument  validity .  In  these  designs,  the  organizational  climate  instrument 
given  at  one  point  in  tins  is  correlated  with  organizational  outcomes  mea¬ 
sured  at  some  future  point  in  time  (usually  3  to  6  months,  although  scam- 
times  as  long  as  3  years;  Likert,  1973).  This  type  of  longitudinal  analysis 
is  accomplished  by  use  of  "cross-lagged  panel  correlations”  in  the  format 
below  (Campbell  S  Stanley,  1967;  Kenny,  1975) t 


Both  the  climate  survey  measure  (standards)  and  the  "hard”  outcome  criteria 
(inspection  scores)  are  assessed  at  two  points  in  time.  Correlations  "a" 
and  "c”  would  provide  an  indication  of  concurrent  validity  between  the  in¬ 
strument  and  the  criteria.  Correlation  ”e,"  if  it  were  significantly  larger 
than  correlation  "f,"  would  provide  evidence  of  predictive  validity  between 
the  instrument  variable  at  one  time  and  the  outcome  variable  at  a  second 
point  in  time.  This  analysis  can  also  establish  the  causal  directionality 
of  the  relationship  between  the  instrument  variable  and  the  criterion. 
(Correlation  coefficients  by  themselves  measure  the  degree  of  association 
between  variables,  but  not  the  directionality  of  the  relationship.)  Causal 
flow  models  such  as  that  illustrated  in  Figure  2  are  based  on  cross- lagged 
panel  studies  of  this  kind. 

Concurrent  and  predictive  validity  correlations  tend  to  be  much  lower 
than  reliability  coefficients;  an  £  of  .33  between  a  variable  measure  and 
a  criterion  is  considered  good  (Ghiselli ,  1966) .  The  amount  of  variance 
in  the  criterion  variable  accounted  for  by  the  instrument  variable  can  be 
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estimated  by  squaring  the  correlation  coefficient.  For  example,  a  validity 
r  of  .33  for  a  work  environment  variable  indicates  that  the  variable  ac¬ 
counts  for  approximately  10  percent  ( . 332  -  .10)  of  the  variance  in  an  or¬ 
ganisational  outcome.  This  may  seem  low,  but  if  an  organisation's  effort 
impacted  on  the  work  environment  effectively  enough  to  cause  a  10  percent 
increase  in  retention  rates,  this  would  be  a  very  significant  outcome  for 
a  military  organisation.  Figure  A-l  illustrates  the  relationship  between  a 
change  in  an  organisational  climate  variable  and  a  change  in  an  outcome 
variable. 

If  the  strength  of  the  relationship  between  a  given  variable  and  a 
performance  outcome  measure  is  known,  an  OESO  consultant  can  estimate 
probable  changes  in  performance  outcomes  from  changes  in  variable  scores 
on  an  organizational  effectiveness  instrument  and  identify  the  most  effi¬ 
cient  placement  of  organizational  intervention  resources  to  bring  about 
desired  changes. 

For  exasq>le,  in  Figure  A-l,  if  an  intervention  (e.g.,  a  team  building 
workshop)  increases  a  climate  variable  (e.g.,  team  spirit)  one  standard 
deviation  (o)  and  it  is  known  that  r_  -  -.7  between  this  team  spirit  vari¬ 
able  and  AMQL  rates,  one  can  predict  that  the  intervention  will  reduce  a 
client  command's  AHOL  rate  by  .7  standard  deviation  (o)  below  the  mean 
AWQL  rate  for  all  ccnmands. 

If  the  cost  of  a  results  variable  can  be  calculated  (e.g.,  the  cost 
of  the  work  lost,  disciplinary  procedures  for  each  person  who  is  AWOL,  or 
the  loss  in  replacement  and  training  dollars  to  the  Army  if  a  junior  offi¬ 
cer  resigns) ,  one  can  further  develop  benefit/cost  and  return  on  investment 
data  for  OE  interventions  byt 

(a) 


(b) 


(c) 


e  B  ■  estimated  benefits  of  OE  intervention  (from  change 
in  climate  variable) 

c  *  cost  of  the  OE  intervention. 


estimating  the  change  in  the  results  variable  from  change  in 
climate  variables; 

calculating  the  dollar  cost  of  the  OE  intervention  using  cost/ 
applied  man  day,  direct  materials,  travel,  and  overhead; 

calculating  the  benefit/cost  return  on  investment  ratio 

where  B/c  *  benefit/cost  ratio  (return  on  investment) 
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Predicted  Change  in  Result 


Figure  A-l.  Predicting  unit  effectiveness  results  from  Organizational 
Clients  Variable  scores. 


Organizational  Climate  Survey  Variable 
M  *  Mean  ANOC.  Rate/Organizational  Climate  Variable  Score  for  all  commands. 


APPENDIX  B 


INSTRUMENTS  REVIEWED 


Instrument 


Reference 


Agency  Climate  Questionnaire 
(ACQ) 


Air  Force  Organizational 


Blake  and  Mouton  (with 
rewarding  and  expansion) 


Business  Organization 
Climate  Index 


Campbell  &  Pritchard 
Instrument 


Employee  Motivation  and 
Morale 


Bartlett,  C.  M.  &  Schneider,  B.  Indi¬ 
vidual  differences  &  organizational 
climate:  Measurement  of  organizational 
climate  by  the  multi-trait,  multi-rater 
matrix.  Personnel  Psychology,  1970,  23, 
493-512. 

Hendrix,  W.  H.  Contingency  approaches 
to  leadership:  A  review  &  synthesis, 
lackland  Air  Force  Base,  TX:  Occupa¬ 
tional  &  Manpower  Research  Division, 

June  1976. 

Greiner,  L.  E.,  Pavlleitch,  D.  P.,  & 
Barnes,  L.  B.  Organizational  climate 
in  a  government  agency. 

Taguiri,  R.  &  Litwin,  G.  H.  (Eds.).  Or¬ 
ganizational  climate:  Explorations  of  a 
concept .  Boston:  Harvard  University 
Press,  1968. 

Mansfield,  R.  &  Payne,  R.  L.  Relation¬ 
ships  of  perceptions  of  organizational 
climate  to  organizational  structure  con¬ 
text  &  hierarchial  position.  Adminis¬ 
trative  Science  Quarterly,  1973,  18, 
515-526. 

Payne,  R.  &  Pheysey,  D.  Stern’s  organi¬ 
zational  climate  index:  A  reconceptuali¬ 
zation  &  application  to  business  organi¬ 
zations  .  Organizational  Behavior  &  Human 
Performance ,  1971,  <3,  77-98. 

Stern/Aston  studies. 

Karasick,  B.  &  Pritchard,  R.  The  effects 
of  organizational  climates  on  managerial 
job  performance  and  job  satisfaction. 
Organizational  Behavior  &  Human  Perfor¬ 
mance,  1973,  9,  110-119. 

Chen,  P.  Some  questionnaire  measures  of 
employee  motivation  and  morale.  Ann 
Arbor:  Institute  for  Social  Research, 
University  of  Michigan,  1975. 
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Instrument 

Encounter  Group  Question¬ 
naires— Leader  Behavior 
Checklist 

Forum  Performance  Audit 


General  Organizational 
Questionnaire  (GOQ) ,  Army 


Halpin  and  Crofts  (adapted) 


Human  Resource  Management 
Survey 


Reference 


Lieberraan,  M. ,  Yalom,  J. £  Miles,  M. 
Encounter  groups:  First  facts.  New 
York:  Basic  Books,  1973. 

Forum  performance  audit.  Boston:  The 
Forum  Corporation,  1974. 

General  organizational  questionnaire 
manual.  Fort  Ord,  CA:  U.S.  Army  Ad¬ 
ministration  Center,  Organizational 
Effectiveness  Training  Center,  1976. 

Organizational  effectiveness  survey  data 
processing  system  (survey)  users  manual 
(interim  version) .  Fort  Ord,  CA:  U.S. 
Army  Administration  Center,  Organiza¬ 
tional  Effectiveness  Training  Center, 
1976. 

Fricdlander,  F.  £  Margulies,  N.  Multi¬ 
ple  impacts  of  organizational  climate  £ 
individual  value  systems  upon  job  satis¬ 
faction.  Personnel  Psychology,  1969,  22 
171-183. 

Bowers,  0.  G.  £  Seashore,  S.  E.  Predict¬ 
ing  organizational  effectiveness  with  a 
four-factor  theory  of  leadership.  Ad¬ 
ministrative  Science  Quarterly,  1966,  11 
238-263. 

Likert,  R.  L. 

Munger,  M. ,  Spencer,  L. ,  £  Thomson,  T. 
The  U.S.  Navy  human  resource  management 
(Shore)  cycle:  A  manual  for  consultants 
Boston:  McBer  £  Co.,  1976. 

Thomas ,  E .  NPRDC . 

Yellow  book.  Boston:  McBer  £  Co. 

Hackman,  J.  R.  £  Lawler,  E.  E. ,  ill.  Em¬ 
ployee  reactions  to  job  characteristics. 
Journal  of  Applied  Psychology  Monograph, 
1971,  55,  259-286. 


Job  Description 
Inventory 


Instrument 


Reference 


Job  Diagnostic  Survey 


Leadermatch  (four 
instruments) 


Leadership  Evaluation  and 
Analysis  Program — U.S. 
Marine  Corps  (LEAP) 


Li  twin  a  Stringer 
(form  1 ) 


Litwin  &  Stringer 


Hackman,  J.  R.  a  Oldham,  G.  R.  An  in¬ 
strument  for  the  diagnosis  of  jobs  and 
the  evaluation  of  job  redesign  projects 
(Technical  Report  No.  4).  New  Haven, 

CT:  Yale  University,  Department  of  Ad¬ 
ministrative  Sciences,  1974. 

Fiedler,  F.  E.  Validation  a  extension  of 
the  contingency  model  of  leadership  ef¬ 
fectiveness:  A  review  of  empirical  find- 
ings.  Psychology  Bulletin,  1971,  76, 
128-148. 


Fiedler,  F.  E.,  Mahan,  a  Chermers .  Lead- 
ermatch.  University  of  Washington,  1976. 


Affourtit,  T.  D.  The  operational  feasi¬ 
bility  of  the  leadership  evaluation  & 
analysis  program  (LEAP) :  Final  report 
(Technical  Report  77-3).  Rockville,  MD: 
Interaction  Research  Institute,  March 
1977. 


Manual  for  the  leadership  analysis  form 
6  LEAP  interaction  inventory.  Washington 
D.C.:  Headquarters ,  U.S.  Marine  Corps, 
1976. 

Downey,  H.  K. ,  Hellriegal,  D. ,  Phelps,  M. 
&  Slocum,  J.  W.  Organizational  climate: 

A  comparative  analysis.  Journal  of  Busi¬ 
ness  Research,  1974,  2,  233-248. 


Empirical  study  combining'  measures  of  or¬ 
ganizational  climate,  organizational  prac¬ 
tices,  job  satisfaction  &  job  performance 


I 


Lafollette,  W.  R.  &  Sims,  H.  P.  Is  satis¬ 
faction  redundant  with  organizational  cli¬ 
mate?  Organizational  Behavior  &  Human 
Performance ,  1975,  13,  257-278. 


Meyer,  H.  H.  Achievement  motivation  & 
organizational  climates.  In  R.  Tagerei  6 
G.  H.  Litwin  (Eds.),  Organizational  cli¬ 
mate:  Explorations  of  a  concept.  Boston 
Harvard  University  Press,  1968. 
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Instrument 


Reference 


NcBer  Organizational 
Inventory 


McBer  Work  Analysis 
Questionnaire 


Measure  of  Military 
Attitudes 


Measure  of  Morale 


Michigan  Organizational 
Assessment  Package 


Military  Company 
Environment  Inventory 


Becklean,  W.  R.  &  Kinkaed,  M.  D.  the 
organizational  audit.  Boston:  Harvard 
University  Graduate  School  of  Business 
Administration,  1968. 

Blake  fi  Norton.  Conflict  resolution. 

Lawrence,  F.  R.  &  Lorsch,  J.  W.  Studies 
in  organizational  design.  Homewood,  IL: 
Richard  D.  Irwin,  Inc.  fi  The  Dorsey  Press, 
1970. 

Litwin,  G.  H.  fi  Stringer,  R.  A.,  Jr. 
Motivation  fi  organizational  climate. 
Cambridge:  Harvard  University  Graduate 
School  of  Business  Administration,  Di¬ 
vision  of  Research,  1968. 

McClelland,  D.  Motives. 

Spencer,  L.  M.  The  McBer  organizational 
inventory — technical  note .  Boston : 

McBer  &  Co.,  1975. 

McClelland.  Social  motives. 

Work  analysis  questionnaire.  Boston : 

McBer  fi  Co.,  1976. 

Bowers,  D.  G. ,  Holz,  R.  F.,  &  Stout,  R. 
Measures  of  military  attitudes.  Arling¬ 
ton,  VA:  U.S.  Army  Research  Institute 
for  the  Behavioral  fi  Social  Sciences,  1977. 

Borman,  W.  C.  The  development  of  measures 
of  morale.  In  O.D.  program:  Instrumenta¬ 
tion,  implementation  &  methodological  is¬ 
sues.  Symposium  presented  at  the  meeting 
of  the  American  Psychological  Association, 
Chicago,  September  1975. 

Michigan  organizational  assessment  package: 
Progress  report  II.  Ann  Arbor:  University 
of  Michigan,  Institute  for  Social  Research, 
August  1975. 

Moos,'  R.  MCET  manual.  Palo  Alto,  CA: 
Stanford  University,  Department  of  Psy¬ 
chiatry,  Social  Ecology  Laboratory,  1974. 


Military  Leadership 
Survey  (MLBS) 


Motivation  £  working 
relations  of  scientists  fi 
engineers  questionnaire 


Occupational  Attitude 
Inventory 


OCDQ  (£  Cattell  Personality 
Inventory /Organizational 
Structure  Instrument) 


OCDQ  £  Profile  Schools 


Organization  Development 
Survey,  Office  of  the 
Deputy  Chief  of  Staff  for 
Personnel,  December  1975 

Organizational  Climate  for 
Schools 


Organizational  Description 
Questionnaire 


Development  of  a  measure  of  Army  leader¬ 
ship  climate:  The  military  leadership 
survey.  U.S.  Army  Research  Institute  for 
Behavioral  £  Social  Sciences,  December 
1974. 

Ohio  studies  on  leadership. 

Pelz,  D.  £  Andrews,  P.  Scientists  in 
organizations:  Productive  climates  for 
research  £  development.  Ann  Arbor : 
University  of  Michigan,  Institute  for 
Social  Research,  1976. 

Occupational  Attitude  Inventory.  Lackland 
Air  Force  Base,  TX:  USAF  Human  Resources 
Laboratory,  May  1975. 

Bishop,  L.  K.  £  George,  J.  R.  Relation¬ 
ship  of  organizational  structure  £ 
teacher  personality  characteristics  to 
organizational  climate .  Administrative 
Science  Quarterly,  1971,  16,  467-475. 

Kali,  J.  w.  A  comparison  of  Halpin  £ 
Croft's  organizational  climates  and 
Likert  fi  Likert's  organizational  systems. 
Administrative  Science  Quarterly,  1972, 

17,  586-590. 

Washington,  D.C.:  Department  of  the  Army, 
DCSPERS,  1975. 


Halpin,  A.  Theory  £  research  in  adminis¬ 
tration.  New  York:  Macmillan  £  Co. , 
1966. 

Burns,  T.  £  Stalker,  G.  The  management 
of  innovation.  London:  Tavistock  Pub¬ 
lications,  Ltd.,  1961. 

House,  R.  J.  fi  Rizzo,  J.  R.  Toward  the 
measurement  of  organizational  practices: 
Scale  development  £  verification.  Jour¬ 
nal  o'f  Applied  Psychology,  1972,  56(5)  , 
388-396. 

McGregor  (Theory  x/y) 
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Instrument 


Reference 
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Organisational  Effectiveness 
Training  Center,  Fort  Ord, 
CA,  1976 

|  Profile  of  Organizational 

Characteristics 

Satisfaction  and  Desire 
Measures,  Alderfer  ERG 


Schneider  &  Hall 


Stern's  College  Charac¬ 
teristics  Index 


Supervisory  Behavior 
Description  (Fleishman) 


Survey  of  Management  Climate 


Hard  Atmosphere  Scale 


Work  Analysis  Questionnaire 


Work  Environment 
Questionnaire 


Likert,  R.  L.  The  human  organization. 

New  York:  McGraw-Hill,  1967. 

Alderfer,  C.  P.  Existence  relatedness 
£  growth.  New  York:  The  Free  Press, 

1972. 

Hall,  D.  T.  &  Schneider,  B.  Toward 
specifying  the  concept  of  work  climate: 

A  study  of  Roman  Catholic  priests. 

Journal  of  Applied  Psychology,  1972, 

56(6),  447-455. 

Murray  &  Lewin 

Stem,  G.  G.  People  in  context:  Measur¬ 
ing  personality,  environment,  congruence 
in  education  S  industry.  New  York:  John 
Wiley  &  Sons,  1970. 

Fleishman,  E.  A  leader  description  for 
industry.  In  Stogdill  £  Coons  (Eds.), 
Leader  behavior:  Its  description  £ 
measurement.  Columbus,  OH:  Ohio  State 
University,  Bureau  of  Business  Research, 
1957. 

Tripex  Industries  (TRIPX) .  Survey  of 
Management  Climate.  Hay  Assoc . ,  1976 . 

Moos,  R.  H.  Evaluating  treatment  envi¬ 
ronments — A  social  ecological  approach. 

New  York:  John  Wiley  £  Sons,  1974. 

Moos,  R.  H.,  &  Horts.  Assessment  of  the 
social  atmospheres  of  psychiatric  wards. 
Abnormal  Psychology,  1968,  20,  442-449. 

Klenp,  G.  0.  Work  analysis  questionnaire. 
Boston:  McBer  fi  Co.,  1976. 

Cohen,  S.  L.  £  Turney,  J.  R.  The  devel¬ 
opment  of  a  work  environment  questionnaire 
for,  the  identification  of  organisational 
problem  areas  in  specific  Armv  work  set¬ 
tings  .  U.S.  Army  ARIBSS,  June  1976. 
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Instrument 

Work  Environment 
Questionnaire — Seneca 
Any  Depot 


Work  Environment  Scale 


Reference 

Work  environment  questionnaire— Seneca 
Army  depot.  Washington#  D.C.:  Depart¬ 
ment  of  the  Army  (Despers) ,  Army  Research 
Institute . 

Insel  &  Moos,  R.  H.  The  work  environment 
scale.  Palo  Alto,  CA:  Stanford  Univer¬ 
sity,  Social  Ecology  Laboratory,  Depart¬ 
ment  of  Psychiatry,  1972. 

Moos,  R.  H.  Evaluating  treatment  environ 
merits.  Hew  York:  John  Wiley  &  Sons, 


APPENDIX  C 


SOURCES  EXAMINED 


Instrument  Source: 

Beslin,  R. ,  Pfeiffer,  J.  W. ,  Instrumentation  in  human  relations  train 
inq.  University  Associates,  1973. 


Instruments : 


General 

Fundamental  Interpersonal  Relations — Behavior 
Fundamental  Interpersonal  Relations — Feeling 
Survey  of  Interpersonal  Values 
Interpersonal  Check  List 
Interpersonal  Rating  Form 

Psychological  Audit  for  Interpersonal  Relations 
The  A-S  Reaction  Study  in  Personality 

Group  Dynamics 

Hill  Interaction  Matrix — B  - 

Hill  Interaction  Matrix — Group 

Reactions  to  Group  Situations 

Group  Leadership  Questionnaire 

Helping  Relationship  Inventory 

Group  Encounter  Survey 

Team  Effectiveness  Survey 

Group  Dimensions  Descriptions  Questionnaire 

Organizational  Climate 

Organization  Health  Survey 
Organizational  Climate 
Educational  Values 
Organizational  Climate  Index 
Organizational  Climate  Questionnaire 


Management/Leadership  Style 
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